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DEFENSE  SCIENCE  2  4  APR  1997 

BOARD 

MEMORANDUM  FOR  THE  UNDER  SECRETARY  OF  DEFENSE  (ACQUISITION  AND 

TECHNOLOGY) 

SUBJECT:  Report  of  the  Defense  Science  Board  Independent  Advisory  Task  Force  on 
FFRDCs  and  UARCs 

I  am  please  to  forward  the  report  of  the  DSB  FFRDC  and  UARC  Independent  Advisory 
Task  Force  (IATF)  which  has  been  chaired  by  Mr.  Gordon  England.  The  IATF  was  charged  with 
reviewing  and  advising  the  Department  on  the  objectives  and  guidelines  for  appropriate  scope  of 
work;  organizational  structure  and  size  of  the  FFRDCs  and  UARCs;  compliance  with  the  annual 
DoD  Management  Plan;  the  sponsor’s  management  processes;  the  level  and  appropriateness  of 
non-DoD  work;  and  the  thoroughness  of  the  FFRDC  five-year  review  process. 

This  IATF  is  an  outgrowth  of  a  recommendation  of  the  1995  DSB  Task  Force  on 
FFRDCs.  Consistent  with  that  earlier  effort,  the  IATF  found  that  DoD  continues  to  have  a  need 
for  FFRDCs  with  special  relationships  with  the  Department. 

The  IATF  recommended  that  the  work  done  by  the  FFRDCs  be  carefully  defined  and 
limited  to  that  work  which  demands  the  special  attributes  of  an  FFRDC,  e.g.,  unique 
competencies.  Other  work  should  be  procured  competitively,  in  a  manner  consistent  with 
acquisition  policy.  The  result  will  be  to  maintain  for  the  Department  the  advantages  of  FFRDCs 
for  certain  work,  while  providing  the  best  available  service  at  the  most  reasonable  price  for  other 
R&D. 


This  recommendation  is  consistent  with  the  current  direction  of  the  Department,  which  has 
resulted  in  a  decline  in  real  terms  in  the  cost  of  man-years  of  technical  support  over  the  past 
several  years,  while  FFRDCs  have  continued  to  deliver  high-quality  work  as  judged  by  their 
customers.  The  IATF  guidance  is  to  encourage  continuation  and  acceleration  of  that  process. 

Because  of  the  unusually  strong  adverse  reactions  to  this  report  by  the  senior  users  of 
FFRDC  products  throughout  DoD,  I  have  attached  your  summary  comments  at  Appendix  O. 

I  encourage  readers  to  consider  both  IATF  recommendations  and  DoD’s  reaction  to  them. 


/I-- 
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I  / 

Craig'Fields 

Chairman 
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November  21,  1996 

MEMORANDUM  FOR  CHAIRMAN,  DEFENSE  SCIENCE  BOARD 

SUBJECT:  Report  of  the  Defense  Science  Board  FFRDC  &  UARC 

Independent  Advisory  Task  Force 

Attached  is  the  initial  report  of  the  DSB  FFRDC  &  UARC 
Independent  Advisory  Task  Force.  The  Task  Force  was 
established  in  November  1995  as  a  standing  body  to  provide 
advice  on  DoD's  management  of  its  11  Federally  Funded 
Research  and  Development  Centers  (FFRDC)  and  6  University 
Affiliated  Research  Centers  (UARC) .  This  report  is  limited 
to  FFRDC s .  The  UARC  review  will  be  conducted  in  the  early 
spring  of  1997,  with  a  report  expected  to  be  published  in  the 
first  half  of  1997. 

The  Task  Force  was  briefed  by  all  of  the  FFRDC s  and 
their  sponsors.  It  also  received  briefings  from  the  OSD, 

GAO,  DoD  IG  and  selected  private  sector  personnel.  The  Task 
Force  visited  several  of  the  FFRDC s  and  reviewed  relevant 
audits  and  studies,  including  previous  DSB  studies  on  the 
FFRDC s .  The  findings  and  recommendations  in  this  report  are 
the  unanimous  conclusions  of  the  Members  of  the  Task  Force. 

In  meeting  its  assigned  responsibilities,  the  Task  Force 
examined  both  DoD  management  policy  level  issues  and 
individual  FFRDC  management  practices.  It  concluded  that  the 
policy  level  issues  are  the  most  important  and  indeed  drive 
the  individual  management  practices.  Accordingly,  this 
initial  report  places  primary  emphasis  on  the  DoD  level 
issues . 

While  the  Task  Force  noted  a  number  of  positive  trends 
(including  recent  cost  reductions  and  maintenance  of  high 
quality) ,  the  Task  Force  also  concluded  that  the  current 
FFRDC  management  system  does  not  provide  the  best  available 
service  at  the  most  reasonable  cost. 

Task  Force  conclusions  are  based  on  findings  that  much 
of  the  work  currently  being  done  in  the  FFRDCs,  while  of  high 
quality,  is  not  of  a  special  R&D  nature  that  demands  an 
FFRDC.  The  DoD  reliance  on  FFRDCs  is  isolating  it  from 
sources  of  new  technology,  and  will  hinder  the  Department's 
ability  to  get  the  best  technical  advice  in  the  future. 


The  Task  Force  developed  several  recommendations  to  deal 
with  the  problems  that  were  identified.  First,  the  Task  Force 
recommends  that  the  work  done  by  the  FFRDCs  be  more  carefully 
defined  and  limited  to  only  those  special  R&D  activities  that 
demand  the  attributes  of  an  FFRDC .  To  accomplish  this,  the 
current  core  work  statements  must  be  significantly  revised. 
Although  some  work  has  already  been  done  in  this  area,  the 
current  core  statements  still  remain  far  too  general. 

The  second  major  recommendation  is  to  insert  competition 
into  the  management  of  this  important  R&D  element.  To  do  this, 
all  the  work  that  will  no  longer  be  in  the  new  core  work 
statements  should  be  competed.  In  cases  in  which  the  revised 
core  work  statements  result  in  less  than  critical  mass  for  an 
FFRDC,  the  FFRDC  should  be  terminated  and  all  of  its  work  should 
be  competed. 

The  third  major  recommendation  addresses  the  implementation 
of  change.  The  Task  Force  recommends  that  the  DoD  effect 
implementation  of  a  competition  transition  plan  by  the  beginning 
of  FY  98  and  that  the  DoD  FFRDC  management  plan  be  revised  to 
conform  to  these  recommended  changes. 

It  is  imperative  that  the  DoD  revise  their  FFRDC  management 
practices.  Competition  and  access  to  broad  commercial  resources 
are  critical  for  the  DoD  to  meet  current  needs  and  to  ensure 
future  military  dominance. 
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Executive  Summary 


Task  Force  Charge 

The  Defense  Science  Board  FFRDC  &  UARC  Independent  Advisory  Task  Force  was 
established  in  November  1995  to  provide  advice  on  DoD’s  management  of  its  1 1  Federally 
Funded  Research  and  Development  Centers  (FFRDC)  and  6  University  Affiliated  Research 
Centers  (UARC).1 

The  Task  Force  is  charged  with  reviewing  and  advising  the  Department  on:  the  objectives  and 
guidelines  for  appropriate  scope  of  work,  organizational  structure,  and  size  of  the  FFRDCs  and 
UARCs;  compliance  with  the  annual  DoD  Management  Plan;  the  sponsor’s  management 
processes;  the  level  and  appropriateness  of  non-DoD  work;  and  the  thoroughness  of  the  FFRDC 
five-year  review  process.2 

The  Task  Force  will  also  periodically  review  selected  FFRDC  and  UARC  programs  and  conduct 
an  independent  semi-annual  review  of  progress  against  recommendations.  This  report  is  limited 
to  FFRDCs.  The  UARC  review  will  be  conducted  in  the  fall  of  1996  and  the  report  published  in 
the  first  half  of  1997. 

This  is  the  initial  report  of  the  Task  Force.  The  findings  and  recommendations  in  this  report  are 
the  unanimous  conclusions  of  the  Members  of  the  Task  Force. 

Background 

The  current  FFRDCs  were  established  to  meet  specific  DoD  needs,  e.g.,  to  provide  operations 
research  and  analyses  support  for  the  Services  (Project  RAND),  the  OSD  and  JCS  (IDA),  or  to 
provide  scientific  and  engineering  support  involving  launch,  space,  and  ground  support  systems. 
When  they  were  created,  the  defined  tasks  for  all  the  FFRDCs  were  such  that  other  competitive 
sources  were  not  available  to  fill  the  need.  For  example,  no  commercial  space  industry  existed  in 
1960  when  Aerospace  was  created  to  meet  a  clear  national  security  need.  There  were,  therefore, 
very  valid  reasons  for  the  establishment  of  special  capabilities  to  address  these  government 
needs.  The  FFRDCs  were  organized  to  address  well-defined  tasks  and  used  their  special 
characteristics  to  meet  government  needs.  Costs,  while  of  interest,  were  not  paramount.  The 
focus  of  FFRDCs  was  on  the  value  of  the  product — for  example,  could  the  Nation  develop  and 
build  a  workable  strategic  defense  against  air  attack?  The  products  were  immediately  used  by  the 
government  clients.  Billions  of  dollars  were  spent  on  the  strategic  air  defense  system  developed 
with  the  assistance  of  Lincoln  Laboratory.  RAND  analysts,  using  operations  research 
techniques,  discovered  and  outlined  the  vulnerability  of  the  U.S.  Air  Force  to  strategic  attack. 
There  was  immediate  feedback  and  further  investigation  to  determine  ways  to  reduce  identified 
vulnerabilities,  to  improve  air  defenses,  and  to  launch  successfully  reconnaissance  satellites  that 


1  A  list  of  Task  Force  members  is  at  Appendix  K. 

2  The  Terms  of  Reference  are  at  Appendix  J. 


provided  some  assurance  against  surprise  attack.  These  were  all  important  needs  and  at  the  time 
constituted  special  research  and  development  activities  that  merited  the  use  of  FFRDCs. 

Findings 

The  situation  that  the  Task  Force  observed  during  its  review  differs  substantially  from  these 
origins.  Much  of  the  work  currently  being  carried  out  in  the  FFRDCs  is  not  special  R&D  that 
cannot  be  done  elsewhere.  For  example,  operations  research  is  applied  to  issues  as  diverse  as 
management  of  aircraft  departure  queues,  optimal  use  of  shared  production  lines  in  a 
manufacturing  plant,  and  setting  capacity  levels  in  service  industries  such  as  car  rental.  Space, 
while  still  a  hostile  environment,  has  spawned  a  robust  U.S.  industry  supporting  both  defense 
and  commercial  activities.  The  commercial  electronics  and  communications  industries  are 
exploding  with  new  information  technologies,  global  communications  networks,  and  applications 
for  data  transfer  and  analysis  that  were  thought  to  be  impossible  only  a  few  years  ago.  The  DoD 
is  faced  not  only  with  a  new  post-Cold  War  security  world,  but  a  new  technology  world  as  well. 

In  this  new  environment,  the  DoD  often  assigns  to  the  FFRDCs  on  a  sole-source,  non¬ 
competitive  basis  work  that  could  be  done  by  others,  either  in  the  government  or  more  likely  by 
the  private  sector.3  The  assignment  often  appears  designed  to  preserve  the  strategic  relationship 
that  is  said  to  exist  between  the  FFRDC  and  its  primary  sponsor,  to  maintain  competency  in 
areas  defined  as  core,  and  ultimately  to  preserve  the  current  organizational  structure.  According 
to  a  1994  DoD  Inspector  General’s  report,  for  example,  sponsors’  justification  for  assigning 
work  includes  the  unique  expertise  or  knowledge  of  the  FFRDC,  ease  and  quickness  of  assigning 
the  work,  and  the  independence  and  objectivity  of  the  FFRDC.4 

Although  at  the  time  of  their  original  establishment  the  individual  FFRDCs  may  have  had  specific 
capabilities  that  allowed  them  to  meet  DoD  needs  most  effectively,  the  current  statements  of 
core  competencies  are  too  broad.  For  example,  strategy  and  doctrine,  force  development  and 
application,  resource  management,  system  development  and  acquisition,  and  technology 
assessment  and  application  are  typical  and  common  to  several  FFRDCs  as  well  as  to  many 
private  sector  and  government  organizations.  The  FFRDCs,  according  to  sponsors’  evaluations, 
do  high-quality  work.  Yet  in  many  cases  the  work  might  be  done  elsewhere — possibly  better. 
The  Task  Force  is  particularly  concerned  about  the  possibility  that  the  FFRDCs  no  longer 
maintain  excellence  in  the  latest  technologies,  particularly  software  and  communications 
technology  that  is  developing  rapidly  in  the  private  sector  spurred  on  by  competition  in  the 
market.  In  such  cases,  the  FFRDCs  may  no  longer  be  the  best  source  of  advice  on  tasks  they 
now  perform. 

In  response  to  concerns  about  costs,  value,  and  the  continued  use  of  the  organizations,  the  DoD 
(and  the  individual  sponsors)  has  developed  an  oversight  process  to  help  ensure  that  FFRDC 
usage  is  appropriate;  however,  this  process  has  become  increasingly  burdensome.  Study  plans 

3  Office  of  Federal  Procurement  Policy  Letter  84-1  (see  Appendix  N)  and  FAR  35.017-1  specifically  prohibit 
FFRDCs  from  competing  for  work. 

Office  of  the  Inspector  General,  Department  of  Defense,  Contracting  Practices  for  the  Use  and  Operations  of 
DoD-Sponsored  Federally  Funded  Research  and  Development  Centers,  Report  No.  95-048,  December  2,  1994,  p. 
11. 
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are  overseen  by  numerous  General  Officers  and  high-level  civil  servants.  The  FFRDCs  are 
subject  to  numerous  audits  and  reviews.  The  management  oversight  and  feedback  mechanisms, 
however,  have  not  focused  hard  enough  on  whether  any  or  all  of  the  work  could  be  accomplished 
better  using  other  sources. 

Conclusions 

The  Task  Force  examined  both  overall  DoD  management  policy  issues  and  specific  management 
practices  that  exist  at  the  FFRDC/Sponsor  level.  Based  on  capabilities  known  to  exist  in 
commercial  firms  in  operations  research,  systems  integration  and  analysis,  technology 
assessment,  and  other  core  competencies  of  the  FFRDCs,5  the  Task  Force  has  concluded  that  the 
current  system  may  not  provide  the  best  available  service  at  the  most  reasonable  cost. 

This  conclusion  reflects  the  Task  Force’s  concerns  that  these  organizations  are  doing  work  that 
might  more  appropriately  be  done  elsewhere,  and  that  because  of  their  special  relationship  with 
sponsors  and  the  lack  of  competition  in  awarding  their  work,  the  FFRDCs  have  insufficient 
incentive  to  innovate  and  maintain  their  access  to  and  awareness  of  technological  innovations  in 
the  private  sector. 

In  its  examination  of  management  policy  and  practices,  the  Task  Force  found  that  many  of  the 
problems  with  specific  management  practices  identified  at  the  FFRDC/Sponsor  level  are 
principally  the  result  of  management  policy  decisions  at  the  DoD  level.  Accordingly,  the  Task 
Force’s  recommendations  emphasize  actions  at  the  DoD  policy  level. 

Recommendations 

The  Task  Force  has  several  recommendations  to  improve  the  current  situation.  These 
recommendations  apply  principally  to  the  Studies  and  Analyses  and  Systems  Engineering  and 
Integration  FFRDCs.  The  Task  Force  will  make  final  recommendations  with  regard  to  the  three 
FFRDC  laboratories  when  it  has  completed  its  review  of  the  UARCs. 

Accurately  Define  FFRDC  Work 

The  Department  must  carefully  define  those  limited  special  R  &D  activities  that  demand  the 
attributes  of  an  FFRDC.  These  tasks  can  continue  to  be  assigned  to  FFRDCs.  All  other  tasks 
that  do  not  require  the  attributes  of  an  FFRDC,  and  for  which  there  are  other  possible  providers, 
should  be  competed.  Congressional  ceilings  have  already  forced  sponsors  to  discriminate  more 
clearly  those  tasks  appropriate  for  FFRDCs;  this  process  needs  to  continue. 

The  reason  for  undertaking  this  problem  redefinition  and  refinement  is  to  ensure  the  availability 
of  the  best  capabilities  in  the  future.  The  DoD  must  manage  all  R&D  capability  to  achieve 


5  Universities  and  major  corporations  have  strong  capabilities  in  these  areas.  Industrial  engineering  emphasizes 
operations  research  and  decision-support  systems;  firms  like  IBM  and  EDS  are  based  on  systems  integration  and 
management;  and  major  high-technology  firms  and  a  large  number  of  specialized  consulting  firms  depend  on 
technology  assessment  and  forecasting  in  making  major  investment  decisions. 

'  '  3 


particular  goals  rather  than  simply  to  ensure  the  survival  of  particular  organizations.  Lincoln 
Laboratory  was  originally  established  to  achieve  a  particular  goal — and  met  that  goal.  MITRE 
was  established  to  implement  the  technology  that  Lincoln  Laboratory  developed  with  regard  to 
strategic  air  defense  and  did  a  fine  job.  These  were  distinct  activities,  well  structured,  with 
specific  goals.  The  original  Studies  and  Analyses  Centers  were  less  focused  on  a  particular  task, 
but  they  too,  at  the  time  they  were  established,  had  special  analytical  skills.  In  general,  that  is  no 
longer  the  case.  The  Department  needs  to  return  to  its  original  approach — use  FFRDCs  to  meet 
its  special  R&D  needs,  and  use  other,  more  effective  capabilities  when  available. 

Many  FFRDCs  receive  funding  and  tasking  from  sources  other  than  their  primary  sponsor.  The 
Task  Force  has  concluded  that  other  government  agency  tasking  through  the  primary  sponsor 
may  be  appropriate  in  some  limited  cases,  although  the  system  needs  to  be  reviewed.  In  many 
cases,  passing  money  through  the  primary  sponsor  results  in  delays  in  funding,  increases  FFRDC 
costs,  and  raises  questions  about  the  importance  of  the  long-term  strategic  relationship  with  the 
primary  sponsor. 

An  important  part  of  developing  an  accurate  definition  of  the  problem  and  identifying  special 
R&D  needs  that  properly  fit  the  definition  for  FFRDC  work  is  restricting  the  core  capabilities  of 
the  FFRDCs  to  what  is  uniquely  needed  from  an  FFRDC.  The  Task  Force  has  carefully  read  the 
core  statements  of  the  centers.  (These  can  be  found  in  Appendices  B  through  I.)  The  statements 
are  too  general  to  be  usefully  applied  in  a  rigorous  examination  of  the  need  to  continue  the 
individual  FFRDCs,  or  to  assign  specific  work  tasks  to  them. 

Revise  the  core  statements  for  each  FFRDC  to  include  only  those  special  R&D  activities 
for  which  there  are  no  alternative  sources.  The  special  R&D  activities  should  be  project 
specific  and  end  result  oriented  rather  than  FFRDC  capabilities  oriented.6  All  included 
activities  must  have  only  RDT&E  funding.  These  new  core  statements  are  to  be  the  basis 
for  continued  contracting  with  the  FFRDCs. 

The  Task  Force  expects  that  the  governmental  function  of  redefining  the  core  capabilities 
statements,  combined  with  accurately  defining  problems,  will  have  the  effect  of  significantly 
reducing  the  size  of  the  FFRDCs  in  almost  all  cases.  In  particular,  the  activities  of  the  Systems 
Engineering/Integration  FFRDCs,  Aerospace  and  MITRE  C3I,  are  expected  to  be  further  reduced 
because  there  are  numerous  other  sources  available  to  do  much  of  the  remaining  work.7  The  Task 
Force  believes  that  properly  defining  the  tasks  best  performed  by  the  FFRDCs  and  competing 
the  remaining  tasks  will  reduce  costs  and  will  provide  DoD  with  better  access  to  new  technology 
and  the  best  advice. 


6  OFPP  Policy  Letter  84-1  requires  that  sponsoring  agreements  include  descriptions  of  the  FFRDCs  mission, 
general  scope  of  effort,  and  its  role  in  furthering  accomplishment  of  the  sponsoring  agency’s  role,  and  be  sufficiently 
descriptive  to  differentiate  from  work  that  should  be  performed  by  a  non-FFRDC.  See  Appendix  N  for  the  full  text. 

7  Other  sources  include  large  systems  integration  firms  such  as  EDS  and  IBM,  as  well  as  a  wide  variety  of  smaller 
firms. 
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Authorize  an  independent  agency,  such  as  the  Defense  Science  Board,  the  OSD  Office  of 
Acquisition  Reform,  or  the  Competition  Advocates  within  the  Services,  to  review  the 
revised  FFRDC  core  statements  to  assure  compliance  with  OFPP  Policy  Letter  84-1. 

It  is  important  that  an  independent  review  of  the  core  statements  and  an  audit  of  work  being  done 
be  accomplished  to  ensure  compliance  with  established  policy.  The  Task  Force  will  also  be 
examining  the  revised  statements  as  a  part  of  its  future  audits. 

Use  Competition 

The  use  of  FFRDCs  results  in  the  absence  of  competition.  The  Task  Force  believes  that  the  lack 
of  competition  in  awarding  tasks  to  FFRDCs  prevents  sponsors  from  having  good  information 
on  the  cost  and  capabilities  of  alternative  sources  to  meet  their  needs.  The  Task  Force  believes 
that  the  lack  of  competition  for  much  of  the  work  currently  being  done  in  the  FFRDCs  is  not 
justified,  nor  in  the  long  run  is  it  in  the  best  interests  of  the  DoD. 

The  Task  Force  considered  several  alternatives  for  managing  this  important  portion  of  the  R&D 
effort.  One  alternative  is  to  continue  to  pursue  DoD’s  current  management  approach  of 
reinforcing  the  rationale  for  the  work  currently  being  done  within  the  construct  of  the  FFRDCs. 
This  rationale  stresses  the  special  relationship  between  the  FFRDCs,  their  sponsors,  and  the  rest 
of  industry,  based  on  their  independence  and  lack  of  bias,  and  the  breadth  of  their  core 
competencies.  The  Task  Force  sees  several  problems  with  this  approach.  One  of  the  most 
important  is  that  the  supporting  rationale  has  become  increasingly  burdensome.  Another 
alternative  is  to  eliminate  the  use  of  the  FFRDC  designation  in  the  DoD  and  open  all  this  work  to 
competition.  Long-term,  special  relationships  would  still  be  sought,  but  through  the  terms  and 
conditions  of  long-term,  competitive  contracts.  The  Task  Force  believes  that  a  middle  ground  can 
emerge  when  the  DoD  refines  the  core  statements,  restricting  the  FFRDCs’  work  programs  to 
match  the  limited  special  R&D  needs  that  exist.  However,  the  remaining  required  FFRDC 
structure  may  be  unsustainable.8 

If  this  is  the  case — that  is,  if  the  current  FFRDCs  shrink  to  a  size  below  a  critical  mass  in  which 
the  work  load  and  funding  available  are  insufficient  to  finance  the  hardware,  software,  and  skills 
required  to  maintain  core  competencies — then  the  DoD  should  consider  free  and  open 
competition  of  all  current  FFRDC  work.  Many  private  sector  organizations  already  have  the 
essential  long-term  relationships  with  clients  and  are  accustomed  to  working  in  open  systems 
which  avoid  bias  toward  particular  vendors  or  technologies  while  focusing  on  meeting  customer 
needs.  Testimony  to  the  Task  Force  by  representatives  from  McKinsey,  a  management 
consulting  firm,  and  Perot  Systems,  a  systems  integration  and  management  firm,  confirmed  that 
they  often  maintain  long-term  relationships  with  major  clients  which  include  access  to  sensitive 
data,  strategic  decisions,  and  technology  developments. 


8  The  1996  Management  Plan  for  FFRDCs  and  UARCs  (see  Appendix  A)  expresses  DoD’s  concern  that  further 
funding  reductions  could  force  these  institutions  below  the  level  needed  to  maintain  critical  core  competencies. 
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Develop  a  competition  transition  plan  by  April  1,  1997  to:  1)  compete  all  current  FFRDC 
work  not  included  in  the  revised  core  statements,  and  2)  if  the  DoD  concludes  that  the 
new  core  statements  result  in  less  than  a  critical  mass  for  an  FFRDC,  terminate  the 
FFRDC  and  compete  all  of  the  current  work. 

Effect  implementation  of  the  competition  transition  plan  by  the  beginning  of  FY98. 
Implementing  the  Change 

The  annual  DoD  FFRDC  Management  Plan  is  the  operative  document  within  the  Department 
which  provides  guidance  to  the  sponsors.9  It  is  important  that  the  Plan  be  revised  to  reflect  these 
recommendations.  Further,  the  potential  for  conflict  of  interest  inherent  in  assigning  the  Office  of 
OSD  Studies  and  FFRDC  Programs  responsibility  both  for  sponsoring  individual  FFRDCs  (IDA 
and  NDRI)  and  for  the  Management  Plan  and  oversight  of  all  FFRDC  activities  needs  to  be 
eliminated.10 

Revise  the  DoD  FFRDC  Management  Plan  to  reflect  the  above  recommendations. 
Eliminate  the  dual  role  responsibilities  of  the  OSD  office  which  both  sponsors  FFRDCs 
and  manages  DoD  FFRDC  activities. 


9  The  most  recent  Management  Plan,  dated  May  1,  1996,  is  reproduced  as  Appendix  A. 

The  Office  of  OSD  Studies  and  FFRDC  Programs  falls  under  Acquisition  Program  Integration,  a  division  within 
the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition  and  Technology  (USD(A&T)).  DDR&E  is  also  within 
USD(A&T). 
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Initial  Report  of  the  Defense  Science  Board  (FFRDC)  &  University  Affiliated  Research 
Center  (UARC)  Independent  Advisory  Task  Force 


OVERVIEW 

Between  the  onset  of  World  War  II  and  1991,  more  than  70  federally  funded  research  centers 
were  established  to  support  the  U.S.  defense  effort.  These  centers  grew  out  of  the  laboratories 
and  research  groups  created  by  the  federal  government  for  defense  research  during  World  War  II. 
Today  eight  agencies,  including  the  DoD,  fund  39  federally  funded  research  and  development 
centers  (FFRDCs).  The  maximum  number  of  such  centers  supporting  the  DoD  in  existence  at 
any  one  time  was  43,  in  1961.  Although  DoD  reviews  have  repeatedly  affirmed  a  continuing 
need  for  such  centers,  many  individual  centers  have  been  either  discontinued  because  they  were 
no  longer  required  or,  far  more  commonly,  decertified  as  FFRDCs  and  continued,  sometimes  on  a 
not-for-profit  basis,  without  the  FFRDC  designation.  Currently,  there  are  1 1  DoD  FFRDCs, 
grouped  into  three  categories:  Studies  and  Analyses  Centers,  Systems  Engineering  and 
Integration  Centers,  and  Laboratories. 


Figure  1:  Obligations  by  FFRDC  Category 
Constant  1995  Dollars 


The  1 1  currently  active  DoD  FFRDCs  are  shown  in  Table  1.  In  FY  95,  they  had  budget 
authority  of  about  $1,253  billion.  They  have  an  overall  budget  limit  established  by  Congress  for 
FY  96  of  $1,163  billion.  This  $1,163  billion  is  also  a  ceiling,  the  total  amount  that  can  be  spent. 
Less  money  might  be  spent.  Funding  trends  over  the  past  decade  are  shown  in  Figure  1. 
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Table  1:  DoD's  Federally  Funded  Research  and  Development  Centers 


Parent 

FFRDC 

organization  ! 

Systems  Engineering  and  Integration  Centers 


Aerospace 


Air  Force 


MITRE 

Corporation 


Subtotal 


Studies  and  Analyses  Centers 


Arroyo  Center 


Project  Air  Force 


National  Defense 
Research  Institute 


IDA-Studies  and  Institute  for 

Analyses/Oper-  Defense  Analyses 

ational  Test  and 
Eval.  Ctr. 


Logistics 

Management 

Institute 


Subtotal 


Research  and  Development  Laboratories 


Lincoln 

Laboratory 


Army 


Air  Force 


OSD 


Navy 


OSD 


Logistics 

Management 

Institute 


Air  Force 


Software 

Engineering 

Institute 


Camegie-Mellon 

University 


Institute  for  National 

Defense  Analyses  Security  Agency 


Subtotal 


Total 


$335 

1,910 

1,765.0 

1,765.0 

$374 

2,109 

1,895.0 

1,895.0 

$709 

4,019 

3,660.0 

3,660.0 

$20 

99 

96.8 

104.0 

$24 

112 

113.0 

113.0 

$19 

105 

103.9 

106.0 

$47 

238 

233.5 

266.0 

$68 

377 

352.5 

382.3 

$29 

166 

148.0 

156.0 

$207 

1,097 

1,047.7 

1,127.3 

$275 

1,018 

920.0 

920.0 

$29 

170 

155.3 

188.7 

$33 

142 

133.5 

150.0 

$337 

1,330 

1,208.8 

1,258.7 

$1,253 

6,466 

5,916.5 

Note:  Command,  Control,  Communication,  and  Intelligence  (C3I);  Office  of  the  Secretary  of  Defense  (OSD). 

MTS  (Members  of  the  Technical  Staff)  includes  the  direct  professional  labor  of  scientists,  engineers,  researchers, 
mathematicians,  analysts,  economists,  and  others  who  perform  professional-level  technical  work.  MTS  is  defined  as  1,810 
hours  of  full-time  professional  effort  and  does  not  include  subcontracted  effort. 

STE  (Staff  Year  of  Technical  Effort)  is  the  new  measure  being  used  by  DoD.  It  is  defined  to  be  1,810  hours  of  paid  effort  for 
technical  services  and  includes  both  FFRDC  employee  and  subcontracted  consultant  technical  effort. 

The  cost  of  both  MTS  and  STE  hours  includes  a  prorated  share  of  support  staff  (secretaries,  technicians,  office  maintenance, 
computer  services,  etc.)  and  therefore  the  MTS  and  STE  totals  do  not  represent  the  total  employment  of  the  FFRDCs. 

Source:  OSD  and  the  General  Accounting  Office. 
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BACKGROUND 


The  current  FFRDCs  are  different  in  size,  funding,  mission,  and  use.  Nine  of  the  centers  are  run 
by  non-profit  corporations.  Two  (SEI  and  MIT  Lincoln  Laboratory)  are  run  by  universities. 

The  MITRE  C3I FFRDC,  Aerospace,  and  MIT  Lincoln  Laboratory  are  all  significantly  larger  and 
funded  at  much  higher  levels  than  the  other  centers. 

Most  of  the  FFRDCs  are  industrially  funded,  meaning  that  within  the  financial  ceilings  imposed 
on  their  DoD  work,  the  actual  work  is  funded  by  the  individual  organizations  within  the  DoD 
who  wish  to  use  their  services.  This  funding  is  typically  passed  through  to  the  FFRDC  via  a 
single  contract  that  the  sponsoring  agency  holds  with  the  FFRDC.  But  some  of  the  FFRDCs  are 
directly  funded  through  budget  line  items.  RAND’s  Project  Air  Force,  for  example,  is  funded  as 
a  line  item  in  the  congressionally  appropriated  budget  for  the  DoD.  The  Arroyo  Center  at 
RAND  and  the  Center  for  Naval  Analyses  are  also  principally  funded  through  a  line  item  in  the 
DoD  budget.  Line  item  funding  is  said  to  have  the  benefit  of  separating  the  funding  decisions 
from  the  immediate  Department  for  which  a  FFRDC  is  doing  studies  although,  as  a  practical 
matter,  the  decisions  on  the  level  of  funding  to  be  requested  (or,  in  the  current  case,  the  level-of- 
effort  made  available)  each  year  still  comes  from  the  sponsoring  agency.  Industrial  funding,  on 
the  other  hand,  is  said  to  have  the  benefit  of  more  closely  associating  the  project  sponsor  with 
the  cost  of  the  project. 

Each  FFRDC  has  an  individual  contract  except  for  the  MITRE  C3I  which  has  two  contracts,  one 
with  the  Air  Force  and  one  with  the  Army.  Several  of  the  FFRDCs  have  multiple  agencies  as 
users,  including  non-DoD  work.  This  multiple  use  appears  to  complicate  oversight  (because  it 
takes  time  to  coordinate  different  objectives  and  schedule  demands  from  multiple  user 
organizations). 

The  government  documents  governing  the  establishment  and  management  of  FFRDCs  do  not 
define  them  in  terms  of  organizational  structure — indeed,  OFPP  Policy  Letter  84-1,  the  basis  for 
DoD  FFRDC  policy,  specifically  notes  that  “FFRDCs  do  not  have  a  prescribed  organizational 
structure.”  Instead,  FFRDCs  are  defined  by  criteria  related  to  the  organizations’  activity  and 
their  relationship  to  a  government  agency.  The  Letter  states  that  all  of  the  following  criteria 
should  be  met  before  an  activity  is  identified  as  an  FFRDC:11 

•  The  activity  should  perform,  analyze,  integrate,  support  (non-financial)  and/or  manage  basic 
research,  applied  research,  and/or  development.  (Specifically  excluded  are  activities  primarily 
engaged  in  routine  quality  control  and  testing,  routine  service  activities,  production,  mapping 
and  surveys,  information  dissemination,  and  performance  of  commercial  activities  [non-R&D] 
which  are  governed  by  OMB  Circular  No.  A-76.) 

•  Performance  is  either  upon  the  direct  request  of  the  Government  or  under  a  broad  charter 
from  the  Government,  and  the  results  are  directly  monitored  by  the  Government. 


11  See  Appendix  N  for  the  full  text  of  OFPP  Policy  Letter  84-1. 
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•  The  majority  of  the  activity’s  financial  support  (70%  or  more)  is  received  from  the 
Government. 

•  In  general,  most  or  all  of  the  facilities  are  owned  by  the  Government  or  funded,  under 
contract,  by  the  Government. 

•  The  activity  is  operated  by  a  university,  consortium  of  universities,  other  non-profit 
organization,  or  an  industrial  firm  as  an  autonomous  organization  or  as  an  identifiable  separate 
operating  unit  of  a  parent  organization. 

•  A  long-term  relationship  evidenced  by  specific  agreement  exists  between  the  operator  of  the 
activity  and  its  primary  sponsor. 

In  addition  to  these  criteria,  the  Letter  notes  that  the  relationship  between  the  activity  and  the 

Government  should  exhibit  several  other  characteristics  in  order  to  qualify  as  an  FFRDC: 

•  The  activity  is  brought  into  existence  at  the  initiative  of  a  Government  agency  or  bureau  to 
meet  some  special  research  or  development  need  which,  at  the  time,  cannot  be  met  as 
effectively  by  existing  in-house  or  contractor  resources. 

•  Work  from  other  than  a  sponsoring  agency  is  undertaken  only  to  the  extent  permitted  by  the 
sponsoring  agency  and  in  accordance  with  the  procedures  of  the  sponsoring  agency. 

•  The  activity  has  access,  beyond  that  which  is  common  to  the  normal  contractual  relationship, 
to  Government  and/or  supplier  data,  employees,  and  facilities  needed  to  discharge  its 
responsibilities  efficiently  and  effectively,  whether  the  data  is  sensitive/proprietary  or  not. 

•  The  primary  sponsor  undertakes  the  responsibility  to  assure  a  reasonable  continuity  in  the 
level  of  support  to  the  activity  consistent  with  the  agency’s  need  for  the  activity  and  the 
terms  of  the  sponsoring  agreement. 

•  The  activity  is  required  to  conduct  its  business  in  a  responsible  manner  befitting  its  special 
relationship  with  the  Government,  to  operate  in  the  public  interest  free  from  organizational 
conflict  of  interest,  and  to  disclose  its  affairs  (as  an  FFRDC)  to  the  primary  sponsor. 

DoD  FFRDC  Management  Structure 

Overall  management  of  the  U.S.  Government’s  FFRDCs  is  governed  by  congressional  legislation 

and  executive  branch  regulation.  The  Office  of  Federal  Procurement  Policy  within  the  Office  of 

Management  and  Budget  (OMB)  establishes  government-wide  policy  on  the  use  and  management 

of  FFRDCs.  As  noted  above,  OFPP  Policy  Letter  84-1  provides  policy  guidance  on  FFRDCs. 

This  guidance  is  implemented  in  FAR  35.017.12 


12  Also  reproduced  at  Appendix  N. 
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The  current  DoD  FFRDC  management  structure  is  outlined  in  Figure  2.  The  Director  of  Defense 
Research  and  Engineering  (DDR&E)  is  responsible  for  developing  overall  policy  for  the  DoD’s 
FFRDCs,  and  in  turn  reports  to  the  Deputy  Secretary  of  Defense  through  the  Under  Secretary  of 
Defense  for  Acquisition  and  Technology.  The  DDR&E  communicates  DoD  policy  and  detailed 
implementing  guidance  to  FFRDC  sponsors  through  a  periodically  updated  management  plan 
(last  updated  1  May  1996;  see  Appendix  A).  The  DDR&E  ensures  that  the  Staff  Years  of 
Technical  Effort  (STE)  are  consistent  with  DoD  policy.  Establishing  STE  levels  essentially 
establishes  the  funding  level  for  each  FFRDC.  Funding  is  guided  by  the  level  of  the  overall 
congressional  ceiling  on  FFRDC  funding  and  DoD  requirements.  The  DDR&E  also  monitors  the 
mechanisms  used  by  the  FFRDC  sponsors  to  ensure  the  appropriateness  and  value  of  the 
FFRDC  activities,  and  oversees  the  implementation  of  the  DoD  FFRDC  Management  Plan. 


Figure  2:  DoD  FFRDC  Management 


Responsibilities 


Selected  Tools 
and  Publications 


♦  Sets  overall  DoD  policy  in  accordance 
with  OFPP  Policy  Letter  84-1  and 
FAR  35 .01 74 

♦  Oversight  of  program 

♦  Allocation  of  ceilings 


DDR&E 


♦  FFRDC  ManagementPlan 
♦  Required  reports  from  sponsors 
4  Contracting  Personnel  Oversight 

♦  DCAA 


♦  Set  Service  and  Agency  FFRDC 
policy  in  accordance  with 
appropriate  documents  from 
above 

♦  Allocation  of  funds  within 
ceilings 

♦  Ensure  the  appropriateness 
and  value  of  FFRDC 
activities 

♦  Effectively  meet 
sponsorneeds  in 
efficient  manner 

♦  Abide  by  stated  rules 


SPONSORS 

Army  Navy  NSA 


♦  Service  Regulations  (eg.  AFR 
20-9,  Army  Regulation  5-21) 
♦  Contracts 
♦  Sponsoring  Agreements 

4  Evaluations  of  quality/cost 
(eg.  annual,  five  year) 

4  Oversight  committees 


4  Internal  management 
documents 
4  Management  control 
4  Procedures 
4  Self  evaluation 


RAND 

Arroyo 

Center 


IDAC3I 

C  S 

k  area  of  circles  is  proportional  to  lunding  of  FFRDC 


Source:  DSB  Task  Force 
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The  OSD,  Military  Services,  and  Defense  Agencies  sponsor  individual  FFRDCs.  The  primary 
sponsor  awards  and  administers  the  5-year  contracts  after  establishing  or  subsequently 
revalidating  the  continued  need  for  the  FFRDC.  Federal  policy  allows  agencies  to  award  these 
contracts  non-competitively. 

The  sponsor  of  each  FFRDC  is  responsible  for  ensuring  that  the  FFRDC  is  being  properly 
utilized,  that  work  being  done  is  consistent  with  the  FFRDC ’s  core  statement,  that  the  costs  of 
the  goods  and  services  are  reasonable,  and  that  products  are  of  high  value.  Sponsors  are 
responsible  for  certifying  the  existence  of  the  special  R&D  need  and  that  there  is  no  alternative 
source  to  meet  that  need.  The  sponsor  also  has  the  responsibility  to  assure  a  reasonable  level  of 
support  to  the  FFRDC,  consistent  with  the  Agency’s  needs  and  the  terms  of  the  Sponsoring 
Agreement. 

The  FFRDCs  and  their  parent  institutions  must  meet  their  clients’  needs  in  an  efficient  manner. 
Unlike  a  non-FFRDC,  a  FFRDC,  brought  into  existence  at  the  initiative  of  the  Government, 
accepts  restrictions  on  its  ability  to  compete  for  other  government  or  commercial  business. 

These  restrictions  are  intended  to  (1)  limit  the  potential  for  conflicts  of  interest  when  FFRDC 
staff  have  access  to  sensitive  government  or  contractor  data,  and  (2)  allow  the  FFRDC  to  form  a 
special,  or  strategic,  relationship  with  its  DoD  sponsor.  The  FFRDCs  have  their  own  set  of 
management  tools  to  ensure  that  these  needs  are  being  met,  that  funds  are  properly  accounted  for 
and  costs  controlled,  and  that  high-quality  products  are  produced. 

The  Task  Force  was  briefed  by  the  OSD  FFRDC  Management  Office  on  the  current  DoD  Action 
Plan  to  implement  controls  on  the  FFRDCs.  The  plan  is  summarized  in  Box  1 . 


Box  1  DoD  ACTION  PLAN 

The  Plan  requires  sponsors  to  define  the  "core”  work  of  the  FFRDCs  in  terms  of  mission,  purpose,  and  capability  specific  to: 

•  Sponsors)  mission  and  operations 

•  Strategic  relationship  with  sponsor(s) 

•  Criteria  for  reviewing/assigning  core  and  non-core  work 

•  Criteria  for  reviewing/approving  non-DoD  work  at  FFRDCs  and  Parent  Institutions 

Sponsors  were  directed  to  identify  inappropriate  non-core  work  and  develop  plans  to  terminate  such  work.  Reports  were  due  to  DDR&E  by 
January  1996. 

'  The  Air  Force  volunteered  to  develop  a  plan  for  experimental  competition  for  some  work  now  performed  by  the  Systems  Engineering  and 
Integration  FFRDCs. 

An  Independent  Advisory  Task  Force  (the  current  Task  Force)  with  the  responsibility  to  review  and  provide  advice  on  DoD’s  FFRDC 
management  was  established. 

The  DoD  defined  "core"  work  levels  using  Management  Plan  guidelines  in  terms  of  Staff  Year  of  Technical  Effort  (STE).  This  procedure 
was  used  to  set  annual  and  projected  core  requirements  that  were  reported  to  Congress. 

The  DoD  plans  to  work  to  remove  Congressionally-imposed  compensation  limitations  unique  to  FFRDCs  to  preclude  degrading  high-quality 
support. 

DoD  plans  to  implement  revised  guidelines  for  management  fee  (these  are  included  in  the  new  Management  Plan).  Changes  include:  moving 
allowable  costs  out  of  fee  and  reducing  fees  accordingly;  establishing  consistent  policies  on  ordinary  and  necessary  costs;  and  revising 
regulations  and  management  plans. 

Finally,  OSD  plans  to  work  to  establish  DoD  control  on  "core"'  as  the  preferred  alternative  to  Congressionally-imposed  fiscal  ceilings  on 
FFRDCs. 

Source:  OSD 
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Recent  DoD  Concerns 


Over  the  years,  there  have  been  a  number  of  concerns  raised  about  the  Government’s  use  of  its 
FFRDCs.  In  recent  years  complaints  have  included  allegations  that  the  FFRDCs  were  doing 
work  that  could  more  appropriately  be  done  by  for-profit  firms.  A  series  of  studies  has 
examined  the  use  of  FFRDCs.13  The  most  recent  DSB  study,  the  Defense  Science  Board  Task 
Force  on  the  Role  of  Federally  Funded  Research  and  Development  Centers  in  the  Mission  of  the 
Department  of  Defense,  reported  its  findings  in  April  1995.  Key  findings  of  that  DSB  report  are 
found  in  Appendix  M. 

One  of  that  Task  Force’s  principal  findings  was  that  there  is  a  lack  of  trust  among  private  sector 
firms  regarding  the  DoD’s  use  of  its  FFRDCs  arising  from  a  lack  of  transparency  in  the 
Department’s  FFRDC  management  process.  To  address  this  trust  issue  that  Task  Force 
reported  that  it  believed  new  management  tools  needed  to  be  put  in  place  to  provide 
transparency  into  the  DoD  decision  processes.  It  recommended  that  an  independent  review 
panel  from  outside  DoD  be  established  to  systematically  review  and  advise  the  Department  on 
its  management,  use  and  oversight  of  its  FFRDCs.  This  Defense  Science  Board  FFRDC  & 
UARC  Independent  Advisory  Task  Force  was  established  as  a  result  of  that  recommendation. 

Task  Force  Terms  of  Reference 

The  Defense  Science  Board  FFRDC  &  UARC  Independent  Advisory  Task  Force  was 
established  in  November  1995  to  provide  advice  on,  and  to  help  establish  guidelines  for,  DoD’s 
management  of  its  Federally  Funded  Research  and  Development  Centers  and  University 
Affiliated  Research  Centers.14 

The  Task  Force  is  specifically  charged  with  reviewing  and  advising  the  Department  on: 

•  the  objectives  and  guidelines  for  appropriate  scope  of  work,  organizational 
structure,  size,  appropriateness  of  customers,  etc.; 

•  compliance  with  the  annual  DoD  Management  Plan  for  FFRDCs  and  UARCs; 

•  the  various  DoD  sponsors’  management  processes  for  FFRDCs  and  UARCs; 

•  the  level  and  appropriateness  of  non-DoD  work,  to  include  work  for  other  than 
the  sponsor,  in  addition  to  non-DoD  work,  undertaken  by  DoD  FFRDCs  and 
UARCs;  and 


13  For  example.  Inspector  General,  Department  of  Defense,  Contracting  Practices  for  the  Use  and  Operations  of 
DoD-Sponsored  Federally  Funded  Research  and  Development  Centers,  Audit  Report  No.  95-048,  December  2, 

1 994;  Office  of  Federal  Procurement  Policy,  Office  of  Management  and  Budget,  Summary  Report  on  the  Adequacy 
of  Federal  Oversight  of  Federally  Funded  Research  and  Development  Centers,  November  1993;  and  Subcommittee, 
on  Oversight  of  Government  Management,  Committee  on  Governmental  Affairs,  United  States  Senate,  Inadequate 
Federal  Oversight  of  Federally  Funded  Research  and  Development  Centers,  July  1992. 

14  A  list  of  Task  Force  members  is  at  Appendix  K.  The  Terms  of  Reference  are  at  Appendix  J. 
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•  the  thoroughness  of  the  FFRDC  Five-Year  Review  process. 

The  Task  Force  will  also  periodically  review  selected  FFRDC  and  UARC  programs  and  conduct 
an  independent  semi-annual  review  of  progress  against  recommendations. 

Scope 

This  is  the  initial  report  of  the  Task  Force.  It  concentrates  on  the  management  of  the 
Department’s  FFRDCs.  The  Task  Force’s  review  of  UARC  management  will  be  conducted  in 
the  fall  of  1996  with  a  report  planned  for  the  first  half  of  1997. 

The  OSD  directive  that  the  Task  Force  review  and  provide  advice  on  appropriate  scope  of  work, 
organizational  structure,  size,  and  appropriateness  of  customers,  mandates  that  the  Task  Force 
examine  the  overall  DoD  management  of  these  important  assets,  as  well  as  the  operations  of 
individual  FFRDCs  and  the  management  activities  of  their  sponsors.  Thus  the  Task  Force  has 
sought  to  understand  the  entire  management  flow  from  the  DDR&E  to  the  FFRDCs.  The  focus 
has  had  the  benefit  of  illuminating  several  broad  DoD  management  policy  issues  that  the  Task 
Force  believes  must  be  addressed,  if  the  work  that  the  FFRDCs  are  currently  doing  is  to  be 
effectively  managed  and  FFRDCs  are  to  continue  to  be  available  in  the  future.  The  Task  Force 
has  divided  its  findings  into  two  categories:  (1)  management  policy  issues  that  affect  the 
overall  structure  and  relationship  of  the  FFRDCs  to  the  DoD  and  their  individual  sponsors,  such 
as  definition  of  core  competencies,  competition,  and  special  relationships,  and  (2)  specific 
management  practices  applied  in  the  more  generalized  FFRDC  management  policy  structure, 
such  as  funding,  oversight,  and  five-year  reviews. 

Approach 

In  performing  its  tasks,  the  Task  Force  reviewed  previous  reports,  regulations  and  directives 
related  to  FFRDC  management,  and  examined  documentation  provided  by  the  FFRDCs  and  their 
government  sponsors.  The  Task  Force  was  briefed  by  all  of  the  FFRDCs’  management  teams, 
by  the  FFRDC  primary  sponsors,  by  some  additional  sponsors  and  by  other  individuals  and 
organizations  with  relevant  information  (Table  2).  The  Task  Force  has  received  excellent 
cooperation  from  the  FFRDCs  and  their  sponsors. 

The  Task  Force  held  its  initial  meeting  in  Washington,  DC  on  December  12-13, 1995.  At  that 
meeting,  the  Task  Force  received  briefings  on  overall  Department  of  Defense  management  and 
oversight  of  the  FFRDC  from  the  OSD  staff,  and  on  specific  FFRDC  management  issues  from 
two  FFRDCs  (Lincoln  Laboratory  and  the  Center  for  Naval  Analyses)  and  their  government 
sponsors.  The  Task  Force  also  heard  from  the  Professional  Services  Council,  an  organization  of 
private  sector  firms.  At  that  meeting  the  Task  Force  made  the  decision  to  visit  as  many  FFRDCs 
as  was  practical. 

The  Task  Force  subsequently  visited  The  Aerospace  Corporation  on  January  31,  1996;  RAND 
Corporation  on  February  1;  and  the  MITRE  Corporation  on  March  1.  A  meeting  was  held  at  the 
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Institute  for  Defense  Analyses  on  April  9.  In  addition  to  IDA  and  its  sponsors,  this  meeting  also 
included  briefings  from  the  Logistics  Management  Institute  (LMI),  the  Software  Engineering 
Institute  (SEI)  and  their  primary  sponsors.  The  Task  Force  also  received  briefings  on  April  9 
from  two  representatives  of  for-profit  management  consulting  firms  engaged  in  providing 
commercial  (non-defense)  strategic  advice  and  support.  The  two  representatives  described  long¬ 
term  strategic  relations  in  the  commercial  sector.  The  Task  Force  held  a  final  FFRDC  wrap-up 
meeting  at  CNA  on  May  22.  At  that  meeting  the  Task  Force  heard  from  representatives  of  the 
DoD  Inspector  General’s  Office  and  the  General  Accounting  Office,  and  received  wrap-up 
briefings  from  several  of  the  FFRDCs.  The  agendas  for  each  of  the  meetings  are  found  at 
Appendix  L. 

Meetings  of  the  Task  Force,  with  the  exception  of  classified  portions  and  Task  Force 
organizational  planning,  have  been  open  to  the  public. 


Table  2:  Meeting  Schedule 


Date 

Location 

Briefers 

12-13  December  1995 

U.S.  Air  Force  Association 

• 

OSD  FFRDC  Office 

Alexandria,  VA 

• 

MIT  Lincoln  Laboratory  and 
primary  sponsor 

• 

Center  for  Naval  Analyses 
and  primary  sponsor 

• 

Professional  Services 

Council 

31  January  1996 

RAND  Corporation 

• 

RAND  Corporation,  its 

Santa  Monica,  CA 

FFRDCs,  and  their  primary 
sponsors 

1  February  1996 

The  Aerospace  Corporation 

• 

Aerospace  Corporation  and 

El  Segundo,  CA 

primary  sponsor 

1  March  1996 

The  MITRE  Corporation 

• 

MITRE  Corporation  and 

Bedford,  MA 

sponsors 

9  April  1996 

Institute  for  Defense  Analyses 

• 

Institute  for  Defense 

Alexandria,  VA 

Analyses,  its  FFRDCs,  and 
primary  and  secondary 

sponsors 

• 

Logistics  Management 
Institute  and  primary  sponsor 

• 

Software  Engineering 

Institute  and  primary  sponsor 

• 

Representatives  from 
commercial  advisory  firms 

22  May  1996 

Center  for  Naval  Analyses 

• 

DoD  Inspector  General 

Alexandria,  VA 

• 

General  Accounting  Office 

• 

Additional  comments  from 
FFRDCs 
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FINDINGS15 
Some  Positive  Trends 


The  consensus  among  those  interviewed  by  the  Task  Force,  both  inside  and  outside  government, 
is  that  the  DoD’s  management  of  its  FFRDCs  has  improved  recently  as  a  result  of  attention  from 
top  level  acquisition  officials.  Sponsors  and  other  users  of  the  FFRDCs  told  the  Task  Force  that 
they  are  pleased  with  the  quality  of  the  work  being  done  by  the  FFRDCs.  They  view  the  work 
assigned  to  the  FFRDCs  as  essential  and,  with  few  exceptions,  of  high  quality. 

The  Task  Force  identified  some  positive  trends.  For  example,  almost  all  the  FFRDCs  report  that 
the  cost  of  Man-Years  of  Technical  Support  (MTS)  has  declined  in  real  terms  over  the  past 
several  years.  These  reductions  appear  to  have  been  the  result  of  FFRDCs  becoming  more 
efficient  in  response  to  congressionally  mandated  ceilings.  The  management  of  several  FFRDCs 
acknowledged  the  connection  and  noted  that  while  they  believed  that  their  organizations  would 
have  become  more  efficient  even  in  the  absence  of  such  ceilings,  the  ceilings  had  focused  their 
attention.  The  FFRDCs  report  an  awareness  of  the  need  for  cost,  as  well  as  quality,  control. 
There  are  a  number  of  cost  control  initiatives  among  the  centers:  restructuring  and  reducing 
management  positions,  redirecting  support  staff,  and  eliminating  non-essential  activities. 

While  costs  of  MTS  have  been  going  down,  the  FFRDCs  and  their  sponsors  report  that  the 
FFRDCs  have  continued  to  deliver  high-quality  work.  An  examination  of  the  most  recent  Five- 
Year  Comprehensive  Reviews  reveals  that  in  several  instances,  according  to  user  surveys,  the 
quality  of  the  work  has  shown  improvement  over  the  period.16  In  their  briefings  to  the  Task 
Force,  however,  the  centers  and  their  sponsors  expressed  concern  about  whether  they  would  be 
able  to  continue  to  provide  the  same  level  of  high-quality  work  in  the  future.  In  particular,  they 
are  concerned  that  funding  ceilings  may  constrain  their  ability  to  attract  and  retain  high-quality 
staff. 

Although  there  are  positive  trends,  the  Task  Force’s  review  of  the  Department’s  overall 
management  of  its  FFRDCs  raised  a  number  of  concerns  that  the  Task  Force  believes  are  either 
not  being  addressed,  even  with  increased  DoD  attention,  or  may  be  worsening  because  of  the 
management  controls  currently  being  instituted  by  the  Department.  Fundamentally,  the  Task 
Force  is  concerned  that  current  management  initiatives,  such  as  changes  in  definition  of  core 
statements  and  use  of  management  fees,  are  either  being  insufficiently  pursued  or  are  dealing 
with  marginal  problems  rather  than  addressing  critical  overall  management  policy  issues,  and 
that  the  current  management  policy  is  ultimately  unsustainable.  These  concerns  are  addressed 
below. 


15 

16 


These  findings  are  also  mapped  according  to  their  relation  to  the  Terms  of  Reference  at  Appendix  J. 
See,  for  example,  survey  results  for  the  Arroyo  Center,  reproduced  in  Appendix  E,  p.  E-24. 
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General  Management  Policy  Concerns 


In  accordance  with  the  Task  Force  Terms  of  Reference,  this  report  addresses  both  broad 
management  policy  issues  and  specific  FFRDC  management  concerns.  As  the  Task  Force 
examined  the  overall  DoD  FFRDC  management  structure  and  considered  individual  activities 
within  individual  FFRDCs,  it  became  clear  that  the  general  management  policy  issues  are  more 
important  than  specific  management  practices. 

The  Task  Force  concurs  that  the  option  for  the  establishment  and  use  of  FFRDCs  by  the 
Department  should  be  retained.  However,  the  Task  Force  is  concerned  that  the  attempt  to  place 
more  activity  than  can  be  justified  under  the  FFRDC  structures  jeopardizes  both  the  work  that 
must  be  done  and  the  institutions  themselves.  Inadequate  discrimination  in  assigning  that  work 
raises  the  burden  placed  on  available  resources.  Moreover,  the  Task  Force  is  concerned  that  the 
FFRDC  structure,  in  which  the  lack  of  competition  limits  incentives  to  maintain  cutting-edge 
capabilities,  is  not  always  the  best  available  supplier  of  the  work  being  done.  For  these  reasons, 
the  Task  Force  believes  that  the  following  management  policy  issues  deserve  immediate 
attention. 

The  Strategic  Relationship 

The  Strategic  Relationship  between  an  FFRDC  and  a  sponsor  has  become  ah  important  element 
of  the  DoD’s  justification  for  the  use  of  its  FFRDCs.  The  FFRDC  strategic  relationship  has 
come  to  be  used  as  shorthand  to  describe  “stable  and  long-term  relationships”17  that  are 
characterized  by  a  depth  of  understanding  of  client  problems,  knowledge  of  client  activities,  and 
discretion  that  is  difficult,  if  not  impossible  to  duplicate.  The  relationship  is  a  prominent  element 
of  the  new  FFRDC  core  capabilities  statements  and  an  important  part  of  the  new  DoD 
Management  Plan.  However,  the  DoD  FFRDC  Core  Definition  Statements  do  not  specify  the 
Department's  “special  research  or  development  needs”  as  identified  in  die  original  justification 
and  as  required  by  OFPP  Policy  Letter  84-1.  The  statements  are  all  structured  to  the  individual 
FFRDC  Mission,  Purpose,  and  Capabilities  with  emphasis  on  their  Strategic  Relationship  and 
Core  Competencies.  While  strategic  relationships  and  core  competencies  are  important  criteria, 
they  do  not  in  and  of  themselves  justify  continuance. 

The  DoD  reports  that  there  are  a  number  of  organizations  (FFRDCs,  UARCs,  and  private  firms) 
having  strategic  relationships  with  the  DoD.  These  are  shown  in  Table  3. 


17  Office  of  the  Secretary  of  Defense,  DoD  Management  Plan  (B,  6,  a). 
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Table  3 


ORGANIZATIONS  WITH  STRATEGIC  RELATIONSHIPS  WITH  DoD 


•  FFRDCs 


-  Studies  &  Analyses:  broadest  charters;  cross-cutting  access  and  advice;  high-level  issues  and 
interest;  must  minimize  conflict  potential 

Laboratories:  fill  specific  voids  where  no  alternative  R&D  sources  exist;  technology  development 
and  transfer  missions 

-  Systems  Engineering:  focus  on  two  mission  areas;  deep  knowledge  of  architectures  and  programs; 
extensive  involvement  in  acquisition 

•  University  Affiliated  Research  Centers  (UARCS) 

University-administered  laboratories  that  focus  on  specific  technologies  and/or  mission  areas 
Similar  to  laboratory  FFRDCs  in  most  cases,  with  some  systems  engineering  functions  and 
missions  (particularly  at  APL-JHU) 

•  For-Profit  Firms 

-  Small  number  of  long-term  relationships  with  divisions  of  for-profit  firms:  narrow  charters;  focus  on 
specific  systems  or  operations;  typically  systems  engineering  and  SETA  functions 

-  Rely  on  contractual  provisions  and  long-term  relationship  to  limit  parent  corporation  involvement  in 
related  areas 

Source:  OSD 


Multiple  studies18  have  indicated  that  the  strategic  relationship  between  an  FFRDC  and  its 
primary  sponsor  is  used  to  justify  more  of  the  work  contracted  to  the  FFRDCs  than  OFPP 
Policy  Letter  84- 1,  the  FAR,  or  customer  requirements  would  justify.  Neither  document  links 
the  long-term  relationship  directly  to  special  access  (although  both  characteristics  are  discussed), 
nor  does  either  document  do  more  than  suggest  that  the  parties  to  the  contract  need  to  have  an 
understanding  that  they  are  engaged  in  a  long-term  commitment  in  order  to  have  the  confidence  to 
invest  the  needed  effort  to  achieve  the  established  goals.  The  strategic  relationship  is  used  in 
some  of  the  Comprehensive  Reviews  to  demonstrate  that  no  other  source  possesses  the  same 
characteristics  as  the  incumbent  (e.g.,  the  alternatives  all  lack  the  strategic  relationship).  In  such 
cases,  the  relationship  is  being  applied  to  justify  the  continued  need  for  a  particular  FFRDC. 

Applied  in  this  manner,  the  strategic  relationship  prevents  consideration  of  alternative  sources  of 
R&D  available  in  the  commercial  marketplace  because  none  of  the  other  potential  providers 
currently  have  the  strategic  relationship.  The  interpretation  of  the  FFRDCs’  relationships  with 
their  sponsors  appears  to  provide  a  permanent  advantage  to  the  incumbent  organizations.  As  a 
result,  the  sponsors  may  overlook  alternatives.  In  today’s  rapidly  changing  technology 
environment,  the  Task  Force  believes  this  stance  is  detrimental  to  the  DoD.  The  Task  Force  was 
briefed  by  personnel  from  commercial  industry  whose  firms  are  engaged  in  non-defense  strategic 
relationships  and  found  that  strategic  relations  similar  to  those  in  the  FFRDCs  are  relatively 


18 

A  consistent  finding  of  many  reviews  of  FFRDCs  over  many  years  is  that  the  review  process  used  to  select  work 
is  often  not  thorough  enough  to  justify  non-competitive  assignment  of  work  to  them.  For  a  summary  of  relevant 
reports,  see,  Appendix  0,  Office  of  the  Inspector  General,  Department  of  Defense,  Audit  Report  No.  95-048,  op.  cit. 
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common.  For  instance,  management  consultants  such  as  McKinsey  and  system  integrators  such 
as  Perot  Systems  and  EDS  have  strategic  long-term  relationships  with  many  of  their 
customers/  clients . 

Competition 

Per  regulation,  FFRDC  services  must  be  procured  non-competitively.  The  Task  Force  is 
concerned  that  the  lack  of  competition  is  having  negative  effects  on  FFRDC  cost  and  quality. 
Lack  of  competition  eliminates  incentives  for  cost  and  performance  improvement  and, 
furthermore,  limits  DoD’s  ability  to  know  whether  other  contractors  could  do  the  work  better  or 
at  less  cost.  OFPP  Policy  Letter  84-1  and  FAR  35.017  preclude  FFRDCs  from  competing  with 
any  non-FFRDC  concern  in  response  to  a  federal  agency  formal  RFP  for  other  than  the  operation 
of  an  FFRDC.  The  FFRDCs  have  generally  been  awarded  their  five-year  contracts  on  a  non¬ 
competitive  basis  (although  Camegie-Mellon  University  won  a  competition  for  SEI  and  the  Air 
Force  advertised  the  RAND  PAF  in  a  Commerce  Business  Daily  solicitation  in  1985).  Recent 
audits  by  the  DoD  Inspector  General  found  inadequate  justification  for  many  of  the  individual 
tasks  assigned  non-competitively  to  FFRDCs.'0  The  Task  Force  has  not  had  time  to  rigorously 
examine  all  of  the  individual  projects  of  the  FFRDCs  for  appropriateness,  but  its  discussions 
with  the  centers  and  sponsors,  and  a  review  of  the  material  provided  on  studies  being  conducted, 
leads  the  Task  Force  to  conclude  that  considerable  work  being  done  at  the  Studies  and  Analyses 
Centers  and  at  the  Systems  Engineering  and  Integration  Centers  does  not  involve  a  special  R&D 
need  and  that  there  are  now  other  sources  to  provide  such  work.  For  instance,  the  Studies  and 
Analyses  Centers  work  on  political  and  military  strategy  that  is  the  stock  in  trade  of  many 
universities  and  for-profit  consulting  firms. 

Another  common  justification  for  awarding  FFRDCs  work  is  ease  and  speed  of  getting  work 
started.  This  justification  was  also  noted  in  the  DoD  Inspector  General’s  audits,  but  not  as  the 
only  reason  for  placing  work  with  the  FFRDCs.20  However,  service  contracts  with  private 
sector  firms  can  have  the  same  ease  and  speed  of  getting  work  started.  For  instance,  Task  Order 
contracts,  competed  in  response  to  specific  needs  and  written  with  funding  ceilings,  can  quickly 
accommodate  new  work  requirements. 

The  failure  to  compete  the  wide  range  of  tasks  now  being  undertaken  by  the  FFRDCs,  but  for 
which  the  centers  do  not  possess  unique  attributes  (other  than  the  established  long-term  strategic 
relationship),  puts  the  DoD  at  a  disadvantage  in  gaining  access  to  new  information.  The  process 
of  receiving  bids  and  proposals  provides  significant  information  on  relevant  capabilities  available 
and  different  approaches  and  technologies  applicable  to  conducting  the  work.  The  lack  of 
competition  in  this  situation  also  ultimately  affects  the  provider  (either  FFRDC  or  non-FFRDC 
firm)  since  it  reduces  the  incentive  to  make  essential  productivity  investments,  and  ultimately 
makes  it  difficult  to  stay  on  the  cutting-edge  of  technological  change. 

The  DoD  has  not  dealt  with  a  fundamental  issue;  namely,  how  can  DoD  rely  on  FFRDCs  for  a 
broad  range  of  technical  advice  when  much  technology  is  being  developed  in  the  competitive 

19  Inspector  General,  Department  of  Defense,  op.cit.,  p.  1 1. 

20  Ibid. 
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private  sector?21  This  challenge  is  as  great  in  the  work  done  by  the  Studies  and  Analyses  Centers 
as  it  is  to  the  Systems  Integration  and  Engineering  Centers  since  rapidly  developing  technology 
affects  communications,  movement,  force  structure,  acquisition,  etc.  The  problem  does  not  exist 
in  areas  where  there  is  a  special  R&D  need  and  no  other  source  is  available,  but  it  does  in  areas 
where  the  work  can  be  done  by  others. 

The  Task  Force  believes  that  the  lack  of  competition  for  much  of  the  work  currently  being  done 
in  the  FFRDCs  is  not  justified,  nor  in  the  long  run  is  it  in  the  best  interests  of  the  country.  The 
lack  of  competition  does  not  allow  the  sponsors  to  have  information  on  the  cost  and  capabilities 
of  alternative  sources  to  meet  the  sponsors’  needs.  The  Task  Force  is  concerned  that  in  the 
absence  of  competition  the  DoD  will  not  receive  information  on  the  potential  impact  of  rapidly 
developing  commercial  technology  opportunities  that  the  DoD  is  attempting  to  exploit  to  meet 
many  of  its  needs. 

As  more  of  the  work  now  awarded  to  FFRDCs  is  competed,  sponsors  may  determine  a  need  to 
protect  the  programmatic  corporate  memory  that  resides  with  the  FFRDC  employees.  Various 
alternatives  should  be  examined  to  protect  that  corporate  memory 22 

Core  Competencies 

The  Task  Force  supports  the  DoD  policy  aimed  at  defining  core  competencies  and  requiring  that 
the  FFRDCs  concentrate  on  core  work.  However,  current  definitions  remain  too  broad  (e.g., 
strategy  and  doctrine,  force  development  and  application,  resource  management,  system 
development  and  acquisition,  technology  application).  (FFRDC  Core  Competencies  are  found  in 
the  Appendices.)  Such  definitions  of  core  competencies  do  not  restrict  the  work  of  FFRDCs  to 
the  specialized  R&D  needs  of  the  DoD  where  no  other  source  is  available.  In  its  discussion  of  its 
management  process  (see  Appendix  E)  the  Army  is  the  most  direct  in  stating  that  with  regard  to 
the  Arroyo  Center,  the  Army’s  only  studies  and  analyses  FFRDC,  whatever  work  is  decided 
upon  for  the  year  is,  by  definition,  core  work.23  The  other  Centers  are  less  direct,  but  the  general 
nature  of  the  current  core  definitions  in  practice  have  the  same  effect.  One  result  is  that  it 
appears  from  both  recent  government  audits,  and  from  the  Task  Force  panel  members’  own 
observations,  that  much  of  the  work  being  done  by  the  FFRDCs  does  not  involve  special 
research  and  development  needs  which  cannot  be  met  as  effectively  by  existing  in-house  or 
contractor  resources,  but  is  necessary  work  (often  not  funded  by  R&D  funds)  that  can  be  done 
by  others.24  Such  work  requires  no  skills  that  are  not  now  available  from  other  sources,  nor  does 
it  require  access  to  special  information  that  could  not  be  supplied  to  others.  The  fact  that  work 


21  Task  Force  observations  here  are  supported  by  the  Department  of  Defense  IG  finding  that  there  is  inadequate 
justification  for  much  of  the  work  assigned  to  the  FFRDCs.  Ibid.,  p.  19. 

'  For  example,  FFRDC  employees  could  be  given  the  same  rights  that  civil  service  personnel  are  given  in  OMB 
Circular  No.  A-76  competitions,  i.e.,  first  right  of  refusal  of  positions  for  which  they  are  qualified  with  the  winning 
contractor. 

23  See  Appendix  E,  p.  E-17. 

The  DoD  Inspector  General  concluded  that  sponsors  did  not  provide  adequate  justification  for  the  non-competitive 
use  of  FFRDCs  for  223  of  the  229  projects  reviewed.  Furthermore,  it  found  that  sponsors  had  not  conducted 
adequate  market  surveys  to  identify  alternative  sources.  See,  Office  of  the  Inspector  General,  op.cit.,  p.  16.  The 
Task  Force  review  of  FFRDC  documentation  confirmed  the  lack  of  adequate  market  surveys  to  identify  alternative 
sources  at  the  time  of  contract  review. 
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that  has  been  above  congressional  ceilings  has  often  been  sent  to  alternative  sources  calls  into 
question  the  uniqueness  of  FFRDC  products. 

While  the  Task  Force  agrees  that  there  are  some  important  tasks  for  which  organizations  with 
FFRDC  characteristics  are  essential,  the  Task  Force  has  concluded  that  the  actual  amount  of  such 
DoD  work  is  smaller  than  that  currently  being  funded  in  the  DoD  FFRDC  programs.  In  contrast 
to  the  situation  decades  ago,  today  operations  research  is  a  rigorous  discipline  in  academia  and 
fundamental  to  industrial  engineering,  there  is  broad  understanding  of  space  technologies,  and 
there  is  an  exploding  commercial  communications  industry.  Further,  many  firms,  such  as  SAIC, 
CTA,  and  other  defense  contractors,  study  force  structure,  defense  acquisition,  and  the  other 
defense  topics  that  comprise  much  of  the  FFRDCs’  current  core  work  statements.  The  sponsors 
acknowledge  this  and  report  that  they  contract  with  these  other  firms  for  all  but  the 
approximately  $1.2  billion  that  goes  to  the  FFRDCs.  But  under  these  circumstances,  the  current 
generalized  core  competencies  are  difficult  to  justify.  The  more  tightly  structured  definitions  of 
some  (e.g.,  IDA’s  Communications  and  Computing  FFRDC  and  MIT  Lincoln  Laboratory) 
appear  more  appropriate. 

If  the  actual  core  competencies  that  constitute  special  R&D  work  that  cannot  be  accomplished 
elsewhere  as  effectively  are  smaller  than  the  current  broad  definitions  suggest,  then  the  DoD  is 
confronted  with  the  challenge  of  maintaining  a  critical  mass  of  talent  for  a  smaller  absolute  core. 
Again,  competition  and  access  to  broad  commercial  resources  are  critical  to  meeting  current  needs 
and  to  ensuring  that  the  resources  are  available  to  meet  future  needs. 

Effects  of  Current  Structure 

The  use  of  the  current  FFRDC  structure  to  accomplish  many  non-FFRDC  tasks  limits  the 
centers’  and  ultimately  their  customers’  access  to  emerging  technologies  and  world-class  technical 
activities.  As  noted  above,  the  Task  Force  is  convinced  that  the  fact  that  FFRDCs  do  not 
compete  in  the  private  sector  technology  market  can  limit  their  ability  to  access  and  understand 
essential  technologies.  This  is  different  from  the  original  concept  of  organizations  meeting  special 
DoD  R&D  needs  unavailable  from  any  other  source. 
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The  Burdens  of  Current  DoD  Policy 

The  DoD’s  development  of  a  management  structure  for  the  current  FFRDC  structure  and 
relationships  (defining  core  capabilities  and  work,  restructuring  fees,  further  restricting 
competition  and  outside  activities),  and  the  oversight  structure  to  ensure  compliance,  is  becoming 
burdensome  for  both  the  sponsors  and  the  FFRDCs.  Without  refined  core  statements  that  limit 
activities  to  only  those  that  represent  a  special  research  and  development  need  that  cannot  be  met 
as  effectively  elsewhere,  additional  layers  of  oversight  will  only  further  detract  from  the 
FFRDCs’  effectiveness. 

In  summary,  while  the  Task  Force  finds  agreement  with  some  of  DoD’s  current  management 
policy  decisions,  the  Task  Force  concludes  that  the  current  policy  changes  are  inadequate  to  meet 
future  DoD  technology  needs  in  a  cost-effective  manner.  The  Task  Force  believes  that  many  of 
their  findings  regarding  specific  management  practices  (described  below)  will  provide  guidance 
that  can  help  the  DoD  transition  into  a  public-private/govemment-institution  relationship  that  is 
more  appropriate  for  the  future. 

Specific  Management  Practices 

Management  Plan 

The  Task  Force’s  judgment  is  that  the  current  DoD  FFRDC  Management  Plan  (see  Appendix  A) 
does  not  adequately  address  priorities  and  goals.  The  Task  Force  is  concerned  over  the  use  of 
Staff  Years  of  Technical  Effort  (e.g.,  level-of-effort)  as  a  planning  factor.  Level-of-effort  is  an 
inappropriate  way  to  manage  FFRDC  activities.  It  creates  incentives  to  maintain  staff  levels, 
thereby  de-emphasizing  productivity  improvements  that  could  be  achieved  through  application 
of  new  technologies. 

The  Report  on  the  Department  of Defense  Five-Year  Plan  for  Federally  Funded  Research  and 
Development  Centers  (FFRDCs)  and  University  Affiliated  Centers  (UARCs)  (see  Appendix  A) 
(hereafter  referred  to  as  the  Five-Year  Plan)  that  the  Task  Force  received  in  late  May  addresses 
some  of  the  concerns  about  the  main  management  plan,  but  perpetuates  some  of  the  management 
problems  identified  by  the  Task  Force.  For  example,  the  Five-Year  Plan  states  that  “the 
expertise  each  FFRDC  offers  the  DoD  is  unique  to  the  needs  of  the  sponsors  and  does  not 
represent  duplication  of  effort  that  can  be  effectively  eliminated  by  consolidation.”  The  Task 
Force  disagrees  with  the  implication  of  the  statement  indicating  that  most  of  the  work  currently 
being  done  in  the  FFRDCs  is  unique.  The  $43  million  in  non-core  work  that  the  Five-Year  Plan 
reports  has  been  identified  and  is  being  transferred  out  of  the  FFRDCs  appears  to  be  only  the 
beginning  of  the  drawdown  that  is  needed.  There  is  now  no  plan  to  achieve  that  drawdown. 
Indeed,  the  Five-Year  Plan  forecasts  a  relatively  stable  future  for  the  FFRDCs  in  terms  of 
projected  level-of-effort  which  discounts  the  possibility  and  need  for  productivity 
improvements. 
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Five-Year  Reviews 


OFPP  Policy  Letter  84-1  specifies  that  FFRDCs  be  reviewed  periodically — a  requirement  met 
by  the  Five-Year  Reviews — to  determine  that  the  sponsor’s  technical  needs  still  exist,  that  no 
alternative  sources  are  available,  and  that  the  FFRDC  is  meeting  the  sponsor’s  needs  effectively 
and  cost  efficiently. 

The  Five-Year  Comprehensive  Reviews  generally  do  not  have  a  sufficient  level  of  analysis  of  the 
continued  need  for  the  work  that  would  be  done  by  an  FFRDC,  the  potential  for  using  a  specific 
alternative  source,  and  the  management  of  costs.  The  Reviews  address  all  the  issues  required  by 
FAR  35.017  and  the  DoD  Management  Plan,  but  the  discussions  are  conducted  in  general  terms 
rather  than  specifics. 

The  objective  criteria  specified  in  OFPP  Policy  Letter  84-1  to  justify  the  establishment  of  an 
FFRDC  are  not  being  applied  in  the  Five-Year  Comprehensive  Reviews  to  justify  the 
continuation  of  an  FFRDC.  The  Task  Force  found  the  principal  criterion  to  be:  “The  activity 
(organization  and/or  facilities)  is  brought  into  existence  at  the  initiative  of  a  Government  agency 
or  bureau  to  meet  some  special  research  or  development  need  which,  at  the  time,  cannot  be  met  as 
effectively  by  existing  in-house  or  contractor  resources.”25  The  continued  need  for  the  work, 
however,  was  usually  discussed  in  terms  of  continued  dangers  in  the  post-Cold  War  period. 

There  were  some  exceptions.  The  MITRE  discussion  was  focused  on  the  expanding  need  for  C3I 
support,  the  importance  of  information  in  modem  warfare  and  the  role  that  MITRE  could  play  in 
assisting  DoD  in  understanding  the  developments.  The  briefings  to  the  Task  Force  at  MITRE 
and  Aerospace,  however,  suggested  to  the  Task  Force  members  the  use  of  less  than  world-class 
technologies.  For  instance,  commercial  capabilities  known  to  Task  Force  members  in  diverse 
areas  such  as  display  technologies,  data  handling  and  transfer  techniques,  and  failure  mode 
analysis  appear  to  be  more  advanced  than  those  in  use  at  MITRE  and  Aerospace. 

The  assessments  of  alternative  sources  are  insufficient.  While  some  reviews  were  better  than 
others  (the  Service  reviews,  for  example,  particularly  the  Army’s,  did  a  better  job  than  did  the 
OSD  reviews),  none  of  the  reviews  examined  the  cost  and  performance  of  specific  alternatives 
even  though  some  specific  opportunities  have  been  identified.26  Only  one  of  the  Reviews,  the 
Air  Force  review  of  Aerospace,  considered  specific  alternatives,  NASA  and  the  Air  Force’s 
Phillips  Laboratory,  but  both  were  determined  to  be  inadequate.  Most  of  the  discussions  were  in 
the  context  of  the  experience  and  long-term  relationship  of  the  current  FFRDC,  and  the  inability 
to  duplicate  that  relationship.  In-house  alternatives  were  dismissed  as  lacking  in  objectivity, 
influenced  by  Service  doctrine,  and  burdened  with  low  pay  for  equal  talent.  For-profit  firms 
were  dismissed  because  of  potential  conflicts  of  interest,  a  lack  of  broad  understanding  of  many 
systems,  and  a  perceived  inability  to  get  data  from  other  competing  sources.  Non-profits  and 
other  FFRDCs  were  dismissed  because  of  a  lack  of  the  broad  base  of  talents  found  in  the  specific 


25  OFPP  Policy  Letter  84-1,  5c,  2a;  see  Appendix  N. 

26  According  to  an  IG  audit,  a  DoD  1988  CBD  announcement  on  renewing  the  IDA  contract  reportedly  brought 
letters  of  interest  from  nine  commercial  firms.  These  firms  were  named  in  the  report  and  might  have  been  contacted 
and  some  assessment  of  costs  and  availability  made.  See,  Office  of  the  Inspector  General,  Department  of  Defense, 
Sole  Source  Justification  for  DoD  Sponsored  FFRDCs,  Report  No.  94-012,  November  4,  1993,  p.  7. 
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FFRDC  being  reviewed  and  lack  of  an  on-going  relationship.  Some  of  these  other  non-profit 
sources,  it  was  acknowledged,  might  take  the  FFRDC ’s  place,  but  at  a  high  cost.  The  alternative 
was  sometimes  dismissed  simply  because  it  was  not  the  current  FFRDC  (the  strategic 
relationship  problem  discussed  earlier). 

Effectiveness  and  efficiency  were  generally  discussed  in  terms  of  how  the  work  was  received  and 
used  by  the  sponsors  and  other  users.  The  discussion  was  generally  directed  toward  the  quality 
of  work  rather  than  the  cost.  For  example,  much  time  is  spent  by  sponsors  in  reviewing  the 
work  accomplished  and  the  work  to  be  assigned.  (The  Air  F orce,  for  instance,  uses  a  decision 
tree  (see  Appendix  G)  for  allocating  tasks  to  Aerospace.)  The  discussions,  however,  show  little 
attention  or  analysis  directed  toward  expected  or  incurred  costs. 

The  evaluation  of  the  FFRDCs’  management  of  cost  effective  operations  were  largely  conducted 
in  terms  of  adherence  to  auditing  rules.  There  were  some  discussions  of  cost  control  activities, 
but  these  were  not  the  center  of  most  of  the  discussions.  The  Task  Force  was  unable  to  obtain 
audited  relative  costs  of  FFRDCs  that  would  allow  comparisons  with  for  profit  organizations. 
Requests  of  the  auditing  agency  indicated  that  no  such  cost  data  exist. 

Funding 

The  Task  Force  found  that  several  of  the  FFRDCs  derive  much  of  their  funding  from  other  than 
the  primary  sponsor.  For  instance,  LMI  received  more  than  20%  of  its  FY  95  funds  from  non- 
DoD  sources  (see  Appendix  D,  p.  D-5).  The  Task  Force  has  concluded  that  other  government 
agency  tasking  through  the  primary  sponsor  may  be  appropriate  in  some  limited  cases,  although 
the  system  needs  to  be  reviewed  because  such  funding  appears  to  increase  management 
complexity  of  oversight  and  raises  questions  about  the  importance  of  the  long-term  strategic 
relationship  with  the  primary  sponsor  that  is  said  to  be  the  basis  for  the  FFRDC  itself. 

OFPP  Policy  Letter  84-1  and  FAR  35.017  specifically  state  that  FFRDCs  should  address  basic 
research,  applied  research,  and/or  development.  Furthermore,  it  is  the  Task  Force  s 
understanding  that,  according  to  federal  appropriation  statute,  only  RDT&E  funds  are  authorized 
to  fund  R&D  activities.  However,  the  Task  Force  found  in  examining  the  funding  sources  of  the 
FFRDCs  that  only  66%  was  RDT&E  funding,  and  RDT&E  funding  does  not  constitute  the 
majority  of  funding  for  some  FFRDCs.  Only  two  FFRDCs,  IDA’s  (C3I)  and  Project  Air  Force, 
were  fully  funded  through  R&D  money.  The  LMI  has  the  smallest  amount  of  R&D  work 
(14.3%)  and  RAND’s  National  Defense  Research  Institute  and  the  Aerospace  Corporation  each 
reported  only  39%  of  their  support  coming  from  R&D  funds.  Indeed,  the  MITRE  C3I  and 
Aerospace,  the  two  largest  Systems  Engineering  and  Integration  Centers,  had  62.5%  and  39% 
R&D  funding  respectively. 

Excessive  Oversight 

The  FFRDC  oversight  systems  of  the  DoD  and  the  various  sponsors  do  not  exhibit  many  of  the 
characteristics  that  the  DoD  is  now  pursuing  in  its  acquisition  reform  initiatives.  The  Task  Force 
was  troubled  by  the  amount  of  time  that  is  spent  by  sponsors,  particularly  for  the  Studies  and 
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Analyses  FFRDCs,  in  meeting  to  assess  work  progress  and  authorize  studies,  and  by  FFRDC 
management  in  responding  to  this  oversight. 

The  Task  Force  is  also  troubled  by  lack  of  focus  on  costs.  While  the  organizations  have  audits 
by  DCAA  and  others  to  ensure  compliance  with  auditing  rules,  cost  evaluation  measures, 
particularly  to  determine  how  FFRDC  costs  compare  to  other  defense  contractors  and  private 
consultants,  are  not  available.  Several  of  the  sponsor  review  procedures  are  reported  in  the 
appendices  (see,  for  example,  Appendix  B  and  Appendix  E). 

Allocation  Process 

The  ceiling  allocation  process  used  by  the  DoD  is  based  on  STEs.  The  latest  version  of  the  DoD 
Management  Plan  states  that  STEs  “shall  be  used  in  sizing  and  managing  DoD-funded  FFRDC 
work.  Although  the  total  number  of  STEs  available  will  be  constrained  by  DoD  budgetary 
considerations  and  statutory  requirements,  STEs  will  provide  a  standard  measure  across  all  of 
DoD’s  FFRDCs  for  projecting  DoD  workload  and  funding  requirements.”  Although  these  and 
other  guidelines  focus  on  maintaining  core  competencies  and  ability  to  perform  core  work,  the 
actual  process  used  to  allocate  STEs  among  the  different  categories  of  FFRDCs  and  individual 
FFRDCs  has  not  been  made  explicit. 

According  to  OSD,  recent  results  of  this  allocation  process  are  that  Studies  and  Analyses  Centers 
have  taken  a  proportionally  smaller  cut  (-14%  between  FY  90-95)  than  have  the  Systems 
Engineering  Centers  (-26%)  or  the  Laboratories  (-36%).  Although  both  sponsors  and  the 
FFRDCs  told  the  Task  Force  that  Systems  Engineering  work  requirements  are  greater  than  these 
cuts  allow,  OSD  reports  that  these  allocation  differences  are  related  to  the  perceived  need  for 
Studies  and  Analyses  activities  to  help  plan  for  the  future.  This  shift  continues  in  the  Five-Year 
Plan.  The  FFRDCs  complained  about  the  process.  The  process  may  not  meet  changing  needs, 
since  it  is  difficult  to  understand  the  rationale  for  the  allocation. 

Independent  Research  and  Development  (IR&D) 

FFRDCs  currently  use  the  IR&D  mechanism  to  finance  some  research.  However,  because  IR&D 
is  used  for  new  business  development,  and  is  interchangeable  with  bid  and  proposal  funds,  the 
Task  Force  believes  that  IR&D  funding  is  inappropriate  for  FFRDCs.  Funding  to  ensure 
continued  expertise  in  a  skill  believed  essential  should  be  direct  and  approved  by  the  sponsor  and 
not  through  the  IR&D  mechanism,  particularly  since  funding  allocation  guidelines  emphasize 
maintenance  of  core  competencies. 

OSD  Staffing 

Having  an  OSD  office  responsible  for  both  developing  overall  FFRDC  policy  and  acting  as  the 
sponsor  for  individual  FFRDCs  can  create  potential  conflicts  of  interest.  The  Office  of  OSD 
Studies  and  FFRDC  Programs,  which  sponsors  IDA  and  NDRI,  falls  under  Acquisition  Program 
Integration,  a  division  within  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology  (USD(A&T)).  However,  the  Office  of  OSD  Studies  and  FFRDC  Programs  is  also 

25 


responsible  to  the  DDR&E  in  developing  overall  FFRDC  policy.  (DDR&E  is  also  within 
USD(A&T).)  This  office  is  responsible  for  checking  over  300  task  orders  per  year  for  its 
sponsored  FFRDCs  to  ensure  the  tasks  are  appropriate,  plus  overall  management  policy  and 
oversight  of  all  eleven  FFRDCs.  It  is  difficult  to  see  how  the  office  could  examine  the  many 
sponsor  tasks  to  ensure  that  they  cannot  be  done  by  other  sources. 

Costs 

The  Task  Force  was  unable  to  make  good  cost  comparisons  between  the  FFRDCs  and  other 
organizations  because  relevant  data  was  not  available.  The  Task  Force  is  concerned  that  cost 
comparisons  are  generally  not  available  (although  some  were  attempted  in  the  Five-Year 
Reviews),  and  that  in  discussions,  sponsors  did  not  exhibit  a  good  knowledge  of  the  costs  of 
products. 

DoD  sponsors  report  that  they  have  generally  not  estimated  the  cost  of  in-house  performance  as 
an  alternative  to  FFRDC  performance.  Nor  have  cost  estimates  typically  been  obtained  for 
private  sector  contract  performance.  Fully  loaded  labor  costs,  for  instance,  are  not  calculated  in  a 
consistent  way  in  the  different  FFRDCs  or  consistent  with  private  contractors,  so  data  are  not 
available  from  DCAA  to  make  meaningful  comparisons. 

Acquisition  regulations  permit  technical  considerations  to  have  more  importance  than  costs  in  the 
selection  of  a  source  for  R&D  performance.  For  non-R&D  performance  the  emphasis  is  on  costs 
of  a  qualified  source.  The  absence  of  cost  estimates  of  alternative  performance  and  the  lack  of 
cost  competition  for  the  $395  million  of  non-R&D  funding,  coupled  with  the  lack  of  meaningful 
performance  metrics  or  emphasis  on  continuous  process  improvement  efforts  by  the  FFRDCs 
and  their  sponsors,  raises  a  serious  question  for  the  Task  Force  as  to  the  cost-effectiveness  of 
current  FFRDCs.  Further,  the  Task  Force  believes  that  the  DoD’s  plan  to  use  an  estimated 
“level-of-effort”  approach  will  do  little  to  control  costs  and  provides  insufficient  incentive  to 
improve  productivity. 

Congressional  Ceiling 

The  Congressional  ceiling  on  FFRDC  funding  appears  to  have  been  an  effective  management  tool 
for  controlling  costs  and  improving  effectiveness.  Cost  curves  show  improved  efficiency  after 
the  imposition  of  the  ceiling.  The  ceiling  has  caused  efficiencies  and  dictated  the  development  of 
priorities  within  individual  FFRDCs.  The  Task  Force  believes  that  a  careful  determination  of 
what  work  continues  to  meet  the  definition  of  special  need  that  cannot  be  met  as  effectively 
elsewhere,  combined  with  the  introduction  of  competition  in  work  that  does  not  meet  the  revised 
core  competencies,  will  allow  the  DoD  to  assure  Congress  that  ceilings  are  no  longer  necessary. 

Other  Agency  Tasking 

Other  agency  tasking  through  the  primary  sponsor  is  appropriate,  although  the  system  needs  to 
be  reviewed.  In  many  cases,  passing  money  through  the  primary  sponsor  results  in  delays  in 
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funding  and  increases  FFRDC  costs.  This  has  been  particularly  true  at  MITRE.  Further, 
multiple  use  at  centers  such  as  the  NDRI  increases  the  complexity  of  the  oversight. 

Acknowledge  Differences 

With  the  limiting  of  work  to  only  that  which  fits  the  definition  for  FFRDC  work  per  regulation, 
refining  of  core  capabilities,  and  the  mandating  that  FFRDCs  do  only  appropriate  work,  the 
FFRDCs  will,  the  Task  Force  believes,  be  reduced  significantly  in  size.  The  impact  on  the 
FFRDCs’  individual  organizations,  and  their  parents,  will  vary.  RAND  Corporation,  for 
example,  noted  that  it  has  a  substantial  non-defense  effort — including  a  fully  accredited  graduate 
school.  Other  parent  organizations  have  fewer  alternatives.  The  DoD  must  carefully  examine  the 
impact  of  their  policies  on  the  viability  of  the  FFRDCs  and  their  parent  organizations. 

SUMMARY 

The  current  FFRDCs  were  established  to  meet  specific  DoD  needs,  e.g.,  to  provide  operations 
research  and  analyses  support  for  the  Services  (Project  RAND),  the  OSD  and  JCS  (IDA),  or  to 
provide  scientific  and  engineering  support  involving  launch,  space,  and  ground  support  systems. 
When  they  were  established,  the  defined  tasks  for  all  these  FFRDCs  were  of  such  a  nature  that 
other  competitive  sources  were  not  available  to  fill  the  need.  For  example,  no  commercial  space 
industry  existed  when  Aerospace  was  created  to  meet  a  clear  national  security  need.  There  were, 
therefore,  very  valid  reasons  for  the  establishment  of  special  capabilities  to  address  these 
government  needs.  The  FFRDCs  were  organized  to  address  well-defined  tasks  and  used  their 
special  characteristics  to  meet  government  needs.  Costs,  while  of  interest,  were  not  paramount. 
The  focus  of  FFRDCs  was  on  the  value  of  the  product — for  example,  could  the  Nation  develop 
and  build  a  workable  strategic  defense  against  air  attack?  The  products  were  immediately  used 
by  the  government  clients.  Billions  of  dollars  were  spent  on  the  strategic  air  defense  system 
developed  with  the  assistance  of  Lincoln  Laboratory.  RAND  analysts,  using  operations  research 
techniques,  discovered  and  outlined  the  vulnerability  of  the  U.S.  Air  Force  to  strategic  attack. 
There  was  immediate  feedback  and  further  investigation  to  determine  ways  to  reduce  identified 
vulnerabilities,  to  improve  air  defenses,  or  to  launch  successfully  reconnaissance  satellites  that 
provided  some  assurance  against  surprise  attack.  These  were  all  important  needs  and  at  the  time 
constituted  special  research  and  development  activities  that  merited  the  use  of  FFRDCs. 

The  situation  that  the  Task  Force  observed  during  its  review  differs  substantially  from  these 
origins.  Much  of  the  work  currently  being  carried  out  in  the  FFRDCs  is  not  special  R&D  that 
cannot  be  done  elsewhere.  For  example,  operations  research  is  applied  to  issues  as  diverse  as 
management  of  aircraft  departure  queues,  optimal  use  of  shared  production  lines  in  a 
manufacturing  plant,  and  setting  capacity  levels  in  service  industries  such  as  car  rental.  Space, 
while  still  a  hostile  environment,  has  spawned  a  robust  U.S.  industry  supporting  both  defense 
and  commercial  activities.  The  commercial  electronics  and  communications  industries  are 
exploding  with  new  information  technologies,  global  communications  networks,  and  applications 
for  data  transfer  and  analysis  that  were  thought  to  be  impossible  only  a  few  years  ago.  The  DoD 
is  faced  not  only  with  a  new  post-Cold  War  security  world,  but  a  new  technology  world  as  well. 
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In  this  new  environment,  the  DoD  often  assigns  to  the  FFRDCs  on  a  sole-source,  non¬ 
competitive  basis  work  that  could  be  done  by  others,  either  in  the  government  or  more  likely  by 
the  private  sector.27  The  assignment  often  appears  designed  to  preserve  the  strategic  relationship 
that  is  said  to  exist  between  the  FFRDC  and  its  primary  sponsor,  to  maintain  competency  in 
areas  defined  as  core,  and  ultimately  to  preserve  the  current  organizational  structure.  According 
to  a  1994  DoD  Inspector  General’s  report,  for  example,  sponsors’  justification  for  assigning 
work  includes  the  unique  expertise  or  knowledge  of  the  FFRDC,  ease  and  quickness  of  assigning 
the  work,  and  the  independence  and  objectivity  of  the  FFRDC.28 

Although  at  the  time  of  their  original  establishment  the  individual  FFRDCs  may  have  had  specific 
capabilities  that  allowed  them  to  meet  DoD  needs  most  effectively,  the  current  statements  of 
core  competencies  are  too  broad.  For  example,  strategy  and  doctrine,  force  development  and 
application,  resource  management,  system  development  and  acquisition,  and  technology 
assessment  and  application  are  typical  and  common  to  several  FFRDCs  as  well  as  to  many 
private  sector  and  government  organizations.  The  FFRDCs,  according  to  sponsors’  evaluations, 
do  high-quality  work.  Yet  in  many  cases  the  work  might  be  done  elsewhere — possibly  better. 
The  Task  Force  is  particularly  concerned  about  the  possibility  that  the  FFRDCs  no  longer 
maintain  excellence  in  the  latest  technologies,  particularly  software  and  communications 
technology  that  is  developing  rapidly  in  the  private  sector  spurred  on  by  competition  in  the 
market.  In  such  cases,  the  FFRDCs  may  no  longer  be  the  best  source  of  advice  on  tasks  they 
now  perform. 

In  response  to  concerns  about  costs,  value,  and  the  continued  use  of  the  organizations,  the  DoD 
(and  the  individual  sponsors)  has  developed  an  oversight  process  to  help  ensure  that  FFRDC 
usage  is  appropriate;  however,  this  process  has  become  increasingly  burdensome.  Study  plans 
are  overseen  by  numerous  General  Officers  and  high-level  civil  servants.  The  FFRDCs  are 
subject  to  numerous  audits  and  reviews.  The  management  oversight  and  feedback  mechanisms, 
however,  have  not  focused  hard  enough  on  whether  any  or  all  of  the  work  could  be  accomplished 
better  using  other  sources. 

CONCLUSIONS 

The  Task  Force  examined  both  overall  DoD  management  policy  issues  and  specific  management 
practices  that  exist  at  the  FFRDC/Sponsor  level.  Based  on  capabilities  known  to  exist  in 
commercial  firms  in  operations  research,  systems  integration  and  analysis,  technology 
assessment,  and  other  core  competencies  of  the  FFRDCs,29  the  Task  Force  has  concluded  that 
the  current  system  may  not  provide  the  best  available  service  at  the  most  reasonable  cost. 


27  Office  of  Federal  Procurement  Policy  Letter  84-1  (see  Appendix  N)  and  FAR  35.017-1  specifically  prohibit 
FFRDCs  from  competing  for  work. 

Office  of  the  Inspector  General,  Department  of  Defense,  Contracting  Practices  for  the  Use  and  Operations  of 
DoD-Sponsored  Federally  Funded  Research  and  Development  Centers,  Report  No.  95-048,  Dec.,  2,  1994,  p.  11. 

Universities  and  major  corporations  have  strong  capabilities  in  these  areas.  Industrial  engineering  emphasizes 
operations  research  and  decision-support  systems;  firms  like  IBM  and  EDS  are  based  on  systems  integration  and 
management;  and  major  high-technology  firms  and  a  large  number  of  specialized  consulting  firms  depend  on 
technology  assessment  and  foresight  in  making  major  investment  decisions. 
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This  conclusion  reflects  the  Task  Force’s  concerns  that  these  organizations  are  doing  work  that 
might  more  appropriately  be  done  elsewhere,  and  that  because  of  their  special  relationship  with 
sponsors  and  the  lack  of  competition  in  awarding  their  work,  the  FFRDCs  have  insufficient 
incentive  to  innovate  and  maintain  their  access  to  and  awareness  of  technological  innovations  in 
the  private  sector. 

In  its  examination  of  management  policy  and  practices,  the  Task  Force  found  that  many  of  the 
problems  with  specific  management  practices  identified  at  the  FFRDC/Sponsor  level  are 
principally  the  result  of  management  policy  decisions  at  the  DoD  level.  Accordingly,  the  Task 
Force’s  recommendations  emphasize  actions  at  the  DoD  policy  level. 

RECOMMENDATIONS 

The  Task  Force  has  several  recommendations  to  improve  the  current  situation.  These 
recommendations  apply  principally  to  the  Studies  and  Analyses  and  Systems  Engineering  and 
Integration  FFRDCs.  The  Task  Force  will  make  final  recommendations  with  regard  to  the  three 
FFRDC  laboratories  when  it  has  completed  its  review  of  the  UARCs. 

Accurately  Define  FFRDC  Work 

The  Department  must  carefully  define  those  limited  special  R  &D  activities  that  demand  the 
attributes  of  an  FFRDC.  These  tasks  should  continue  to  be  assigned  to  FFRDCs.  All  other 
tasks  that  do  not  require  the  attributes  of  an  FFRDC  as  stated  and  required  in  OFPP  Policy 
Letter  84-1  and  the  FAR,  and  for  which  there  are  other  possible  providers,  should  be  competed. 
Congressional  ceilings  have  already  forced  sponsors  to  discriminate  more  clearly  those  tasks 
appropriate  for  FFRDCs;  the  process  needs  to  continue. 

The  reason  for  undertaking  this  problem  redefinition  and  refinement  is  to  ensure  the  availability 
of  the  best  capabilities  in  the  future.  The  DoD  must  manage  all  R&D  capability  to  achieve 
particular  goals  rather  than  simply  ensuring  the  survival  of  a  particular  organization.  Lincoln 
Laboratory  was  originally  established  to  achieve  a  particular  goal — and  met  that  goal.  MITRE 
was  established  to  implement  the  technology  that  Lincoln  Laboratory  developed  with  regard  to 
strategic  air  defense  and  did  a  fine  job.  But  these  were  distinct  activities,  well  structured,  with 
specific  goals.  The  original  Studies  and  Analyses  Centers  were  less  focused  on  a  particular  task, 
but  they  too,  at  the  time  they  were  established,  had  special  analytical  skills.  In  general,  that  is  no 
longer  the  case.  The  Department  needs  to  return  to  this  original  approach— use  FFRDCs  to  meet 
special  R&D  needs,  and  use  other,  more  effective  capabilities  for  other  needs. 

Many  FFRDCs  receive  funding  and  tasking  from  sources  other  than  their  primary  sponsor.  The 
Task  Force  has  concluded  that  other  government  agency  tasking  through  the  primary  sponsor 
may  be  appropriate  in  some  limited  cases,  although  the  system  needs  to  be  reviewed.  In  many 
cases,  passing  money  through  the  primary  sponsor  results  in  delays  in  funding,  increases  FFRDC 
costs,  and  raises  questions  about  the  importance  of  the  long-term  strategic  relationship  with  the 
primary  sponsor. 
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An  important  part  of  developing  an  accurate  definition  of  the  problem  and  identifying  special 
R&D  needs  that  properly  fit  the  definition  for  FFRDC  work  is  restricting  the  core  capabilities  of 
the  FFRDCs  to  what  is  uniquely  needed  from  an  FFRDC.  The  Task  Force  has  carefully  read  the 
core  statements  of  the  centers  (see  Appendices  B  through  I).  The  statements  are  too  general  to 
be  usefully  applied  in  a  rigorous  examination  of  the  need  to  continue  the  individual  FFRDCs,  or 
to  assign  specific  work  tasks  to  them. 

Revise  the  core  statements  for  each  FFRDC  to  include  only  those  special  R&D  activities 
for  which  there  are  no  alternative  sources.  The  special  R&D  activities  should  be  project 
specific  and  end  result  oriented  rather  than  FFRDC  capabilities  oriented.30  All  included 
activities  must  have  only  RDT&E  funding.  These  new  core  statements  are  to  be  the  basis 
for  continued  contracting  with  the  FFRDCs. 

The  Task  Force  expects  that  the  governmental  function  of  redefining  the  core  capabilities 
statements,  combined  with  accurately  defining  problems,  will  have  the  effect  of  significantly 
reducing  the  size  of  the  FFRDCs  in  almost  all  cases.  In  particular,  the  activities  of  the  Systems 
Engineering/Integration  FFRDCs,  Aerospace  and  MITRE  C3I,  are  expected  to  be  further  reduced 
because  there  are  numerous  other  sources  available  to  do  much  of  the  remaining  work.31  The 
Task  Force  believes  that  properly  defining  the  tasks  best  performed  by  the  FFRDCs  and 
competing  the  remaining  tasks  will  reduce  costs  and  will  provide  DoD  with  better  access  to  new 
technology  and  the  best  advice. 

Authorize  an  independent  agency,  such  as  the  Defense  Science  Board,  the  OSD  Office  of 
Acquisition  Reform,  or  the  Competition  Advocates  within  the  Services,  to  review  the 
revised  FFRDC  core  statements  to  assure  compliance  with  OFPP  Policy  Letter  84-1. 

It  is  important  that  an  independent  review  of  the  core  statements  and  an  audit  of  work  being  done 
be  accomplished  to  ensure  compliance  with  established  policy.  The  Task  Force  will  also  be 
examining  the  revised  statements  as  a  part  of  its  future  audits. 

Use  Competition 

The  use  of  FFRDCs  results  in  the  absence  of  competition.  The  Task  Force  believes  that  the  lack 
of  competition  in  awarding  tasks  to  FFRDCs  prevents  sponsors  from  having  good  information 
on  the  cost  and  capabilities  of  alternative  sources  to  meet  their  needs.  The  Task  Force  believes 
that  the  lack  of  competition  for  much  of  the  work  currently  being  done  in  the  FFRDCs  is  not 
justified,  nor  in  the  long  run  is  it  in  the  best  interests  of  the  DoD. 

The  Task  Force  considered  several  alternatives  for  managing  this  important  portion  of  the  R&D 
effort.  One  alternative  is  to  continue  to  pursue  DoD’s  current  management  approach  of 


30  OFPP  Policy  Letter  84-1  requires  that  sponsoring  agreements  include  descriptions  of  the  FFRDCs  mission, 
general  scope  of  effort,  and  its  role  in  furthering  accomplishment  of  the  sponsoring  agency’s  role,  and  be  sufficiently 
descriptive  to  differentiate  from  work  that  should  be  performed  by  a  non-FFRDC.  See  Appendix  N  for  the  full  text. 

Other  sources  include  large  systems  integration  firms  such  as  EDS  and  IBM,  as  well  as  a  wide  variety  of  smaller 
firms. 
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reinforcing  the  rationale  for  the  work  currently  being  done  within  the  construct  of  the  FFRDCs. 
This  rationale  stresses  the  special  relationship  between  the  FFRDCs,  their  sponsors,  and  the  rest 
of  industry,  based  on  their  independence  and  lack  of  bias,  and  the  breadth  of  their  core 
competencies.  The  Task  Force  sees  several  problems  with  this  approach.  One  of  the  most 
important  is  that  the  supporting  rationale  has  become  increasingly  burdensome.  Another 
alternative  is  to  eliminate  the  use  of  the  FFRDC  designation  in  the  DoD  and  open  all  this  work  to 
competition.  Long-term,  special  relationships  would  still  be  sought,  but  through  the  terms  and 
conditions  of  long-term,  competitive  contracts.  The  Task  Force  believes  that  a  middle  ground  can 
emerge  when  the  DoD  refines  the  core  statements,  restricting  the  FFRDCs’  work  programs  to 
match  the  limited  special  R&D  needs  that  exist.  However,  the  remaining  required  FFRDC 
structure  may  be  unsustainable.32 

If  this  is  the  case — that  is,  if  the  current  FFRDCs  shrink  to  a  size  below  a  critical  mass  in  which 
the  work  load  and  funding  available  are  insufficient  to  finance  the  hardware,  software,  and  skills 
required  to  maintain  core  competencies — then  the  DoD  should  consider  free  and  open 
competition  of  all  current  FFRDC  work.  Many  private  sector  organizations  already  have  the 
essential  long-term  relationships  with  clients  and  are  accustomed  to  working  in  open  systems 
which  avoid  bias  toward  particular  vendors  or  technologies  while  focusing  on  meeting  customer 
needs.  Testimony  to  the  Task  Force  by  representatives  from  McKinsey,  a  management 
consulting  firm,  and  Perot  Systems,  a  systems  integration  and  management  firm,  confirmed  that 
they  often  maintain  long-term  relationships  with  major  clients  which  include  access  to  sensitive 
data,  strategic  decisions,  and  technology  developments. 

Develop  a  competition  transition  plan  by  April  1, 1997  to:  1)  compete  all  current  FFRDC 
work  not  included  in  the  revised  core  statements,  and  2)  if  the  DoD  concludes  that  the 
new  core  statements  result  in  less  than  a  critical  mass  for  an  FFRDC,  terminate  the 
FFRDC  and  compete  all  of  the  current  work. 

Effect  implementation  of  the  competition  transition  plan  by  the  beginning  of  FY98. 
Implementing  the  Change 

The  annual  DoD  FFRDC  Management  Plan  is  the  operative  document  within  the  Department 
which  provides  guidance  to  the  sponsors.33  It  is  important  that  the  Plan  be  revised  to  reflect 
these  recommendations.  Further,  the  potential  for  conflict  of  interest  inherent  in  assigning  the 
Office  of  OSD  Studies  and  FFRDC  Programs  responsibility  both  for  sponsoring  individual 
FFRDCs  (IDA  and  NDRI)  and  for  the  Management  Plan  and  oversight  of  all  FFRDC  activities 
needs  to  be  eliminated.34 


32  The  1996  Management  Plan  for  FFRDCs  and  UARCs  (see  Appendix  A)  expresses  DoD’s  concern  that  further 
funding  reductions  could  force  these  institutions  below  the  level  needed  to  maintain  critical  core  competencies. 

33  The  most  recent  Management  Plan,  dated  May  1 ,  1 996,  is  reproduced  as  Appendix  A. 

34  The  Office  of  OSD  Studies  and  FFRDC  Programs  falls  under  Acquisition  Program  Integration,  a  division  withm 
the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition  and  Technology  (USD(A&T)).  DDR&E  is  also  within 
USD(A&T). 
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Revise  the  DoD  FFRDC  Management  Plan  to  reflect  the  above  recommendations. 
Eliminate  the  dual  role  responsibilities  of  the  OSD  office  which  both  sponsors  FFRDCs 
and  manages  DoD  FFRDC  activities. 
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Appendix  A 


DEPARTMENT  OF  DEFENSE 
FEDERALLY  FUNDED  RESEARCH  AND  DEVELOPMENT 
CENTER  (FFRDC)  MANAGEMENT  PLAN 
May  1, 1996 


INTRODUCTION 


DoD-sponsored  Federally  Funded  Research  and  Development  Centers  (FFRDCs)  represent  a 
long-term  Government  investment  in  a  unique  resource  for  research,  systems  engineering,  and 
analysis.  Over  the  years  FFRDCs  have  been  essential  contributors  to  maintaining  the  superiority 
of  United  States  forces.  FFRDCs  perform  work  that  is:  (1)  consistent  with  the  center’s  mission, 
purpose  and  capabilities;  (2)  consistent  with  DoD’s  needs  as  reflected  in  the  center  s  core 
competencies;  (3)  consistent  with  the  strategic  relationship  between  the  center  and  its  sponsors, 
and,  (4)  cannot  be  performed  as  effectively  by  existing  in-house,  other  not-for-profit,  or  for-profit 
contractor  resources. 

Because  of  the  importance  and  unique  status  of  FFRDCs,  the  DoD  must  ensure  that  their  use  is 
appropriate  and  that  DoD  has  effective  policies  and  procedures  for  their  management. 

A.  PURPOSE 

This  plan  defines  DoD  policies  and  procedures  for  the  management  and  use  of  DoD-sponsored 
FFRDCs.  It  also  provides  guidelines  and  procedures  for  ensuring  compliance  with  the 
Government-wide  policies  set  forth  in  Office  of  Federal  Procurement  Policy  (OFPP)  Policy  Letter 
84-1,  Federally  Funded  Research  and  Development  Centers,  dated  April  4,  1984,  and  Federal 
Acquisition  Regulation,  Part  35.017,  entitled  Federally  Funded  Research  and  Development 
Centers. 

B.  APPLICABILITY 

1 .  This  plan  applies  to  FFRDC  primary  sponsors,  FFRDC  users,  and  contracting  activities 
that  award  FFRDC  contracts. 

2.  DoD  FFRDCs  provide  high-quality  research,  systems  engineering  and  analytical  work  that 
is  within  the  scope  of  their  defined  core  work  and  draws  on  or  sustains  the  strategic  relationship 
between  the  FFRDC  and  its  sponsor. 

3.  In  addition  to  meeting  long-term  and  intermediate  needs  of  sponsors  and  users,  FFRDCs 
also  provide  immediate,  short-term  assistance  to  address  urgent  high  priority  issues. 


4.  DoD  FFRDCs  are  currently  operated  by  universities  or  privately  organized,  not-for-profit 
corporations  through  long-term  Government  contracts  under  the  authority  of  10  U.S.C.  2304(c) 
(3)(B). 

5.  FFRDCs  are  outside  the  Government  to  permit  the  management  flexibility  necessary  to 
attract  and  retain  high-quality  scientists,  engineers,  and  managers,  and  to  provide  an  independent 
perspective  on  the  critical  issues  they  address  for  their  sponsors  and  users. 

6.  The  nature  of  their  mission  requires  that  FFRDCs  operate  in  a  strategic  relationship  with 
their  sponsors  and  users.  Strategic  relationships  have  the  following  characteristics: 

a.  FFRDCs  and  sponsors  commit  to  stable  and  long-term  relationships. 

b.  FFRDCs  are  granted  access  to  Government  and  contractor  information  beyond  that 
which  is  common  to  the  normal  contractual  relationship,  including  intelligence  data  and  program 
planning  information. 

c.  FFRDCs  bear  a  special  responsibility  to  avoid  actual  and  perceived  conflicts  of 
interest,  and  they  accept  stringent  restrictions  on  their  scope,  method  of  operations  and  the  kinds 
of  efforts  they  can  undertake  either  for  their  sponsors  or  for  other  users. 

7.  Strategic  relationships  enable  FFRDCs  to: 

a.  Develop  and  maintain  in-depth  knowledge  of  their  sponsors’  programs  and  operations. 

b.  Maintain  continuity  and  currency  in  their  special  fields  of  expertise,  and  a  high  degree 
of  competence  in  their  staff  and  work. 

c.  Maintain  their  objectivity  and  independence. 

d.  Respond  to  the  emerging  needs  of  their  sponsors  and  users. 

8.  The  DoD  currently  sponsors  eleven  FFRDCs  managed  by  eight  parent  organizations  (see 
Appendix  A).  Each  of  the  eleven  FFRDCs  fall  under  one  of  the  three  categories  of  FFRDCs 
defined  by  the  National  Science  Foundation.  This  management  plan  recognizes  the  different 
purposes  and  contributions  by  organizations  in  each  category.  The  distinctions  between 
categories  of  FFRDCs  are  an  important  consideration  in  the  management  approach  that  should  be 
applied  to  each  of  them.  The  three  categories  as  represented  in  the  DoD  are: 

a.  Studies  and  Analyses  (S&Aj  Centers:  S&A  centers  deliver  independent  and  objective 
analyses  and  advice  in  core  areas  important  to  their  sponsors  in  support  of  policy  development, 
decision  making,  alternative  approaches,  and  new  ideas  on  issues  of  significance. 

b.  Systems  Engineering  and  Integration  fSE&Il  Centers:  SE&I  centers  provide  required 
support  in  core  areas  not  available  from  sponsor's  in-house  technical  and  engineering  capabilities 
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to  ensure  that  complex  systems  will  meet  operational  requirements.  The  centers  assist  with  the 
creation  and  choice  of  system  concepts  and  architectures,  the  specification  of  technical  system  and 
subsystem  requirements  and  interfaces,  the  development  and  acquisition  of  system  hardware  and 
software,  the  testing  and  verification  of  performance,  the  integration  and  interoperability  of  new 
capabilities,  and  continuous  improvement  of  system  operations  and  logistics.  They  often  play  a 
critical  role  in  assisting  their  sponsors  in  technically  formulating,  initiating,  and  evaluating 
programs  and  activities  undertaken  by  firms  in  the  for-profit  sector. 

c.  Research  &  Development  tR&Dt  Laboratories:  R&D  laboratories  fill  voids  where  in- 
house  and  private  sector  research  and  development  centers  are  unable  to  meet  DoD  core  area 
needs.  Specific  objectives  for  these  FFRDCs  are  to:  (1)  maintain  over  the  long-term  a 
competency  in  technology  areas  where  the  Government  cannot  rely  on  in-house  or  private  sector 
capabilities;  and  (2)  develop  and  transfer  important  new  technology  to  the  private  sector  so  the 
Government  can  benefit  from  a  wider,  broader  base  of  expertise.  R&D  laboratories  engage  in 
research  programs  that  emphasize  the  evolution  and  demonstration  of  advanced  concepts  and 
technology,  and  the  transfer  or  transition  of  technology'. 

C.  DEFINITIONS 

1 .  Primary  Sponsor.  A  specified  DoD  organization  will  be  designated  as  a  primary  sponsor 
by  the  Director  Defense  Research  and  Engineering  (DDR&E)  for  each  FFRDC  (see  Appendix  A). 
The  primary  sponsor  is  responsible  for  implementing  FFRDC  management  policies  and 
procedures. 

2.  Contracting  Activity.  As  referred  to  in  this  plan,  the  DoD  contracting  activity  is  the 
activity  that  awards  a  contract  or  contracts  under  the  authority  of  10  U.S.C.  2304  for  a  FFRDC. 

3 .  User.  The  user  or  tasking  activity  is  an  entity  (DoD  or  non-DoD)  that  requires  the 
services  of  a  DoD  FFRDC  for  performance  of  FFRDC  work. 

4.  Core  Work.  Work  appropriate  for  performance  by  a  FFRDC  because  it  is  consistent  with 

the  mission,  purpose  and  competencies  of  the  FFRDC,  and  draws  on  or  sustains  a  strategic 
relationship  between  the  FFRDC  and  its  sponsor. 

5  Non-FFRDC  Work.  Work  performed  by  the  parent  institution  that  does  not  comply  with 
the  definition  of  core  work,  and  is,  therefore,  not  performed  within  the  FFRDC(s). 

6.  Parent  Institution.  An  entity  that  contracts  with  the  DoD  to  operate  a  FFRDC.  A  parent 
institution  may  also  be  called  a  parent  organization. 


D.  POLICY 
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1.  Policy  for  the  Performance  of  Work  bv  DoD-Sponsored  FFRDCs  and  Parent  Institutions. 
DoD  sponsors  must  operate  under  this  policy.  Specific  implementing  instructions  shall  be 
documented  in  the  respective  sponsoring  agreement. 

a.  FFRDC  Work:  A  DoD  FFRDC  may  only  perform  core  work  as  defined  in  its  core 
statement  and  in  accordance  with  the  following  guidelines: 

(1)  All  work  must  be  approved  by  the  primary  sponsor. 

(2)  Work  may  only  be  accepted  from  DoD,  other  Government  entities,  state  and 
municipal  governments,  and  not-for-profit  activities. 

(3)  No  commercial  work  may  be  accepted  by  a  DoD  FFRDC. 

b.  Non-FFRDC  Work:  Parent  institutions  operating  DoD  FFRDC(s)  may  perform  non- 
FFRDC  work  subject  to  primary  sponsor  review  for  compliance  with  established  criteria  mutually 
agreed  upon  by  the  primary  sponsor  and  parent  institution.  The  criteria  shall  be  addressed  in  the 
Sponsoring  Agreement.  In  establishing  these  criteria,  the  following  guidelines  shall  be  used  by  the 
primary'  sponsor: 

(1)  Non-FFRDC  work  by  parent  institutions  should  be  in  the  national  interest,  such  as 
addressing  economic,  social,  or  governmental  issues. 

(2)  Non-FFRDC  work  shall  not  undermine  the  independence,  objectivity,  or  credibility 
of  the  FFRDC  by  posing  an  actual  or  perceived  conflict  of  interest,  nor  shall  it  detract  from  the 
performance  of  FFRDC  work. 

(3)  Non-FFRDC  work  shall  not  be  acquired  by  taking  unfair  advantage  of  the  parent 
institution’s  operation  of  its  FFRDC(s)  or  of  information  that  is  available  to  that  parent  institution 
only  through  its  FFRDC(s). 

(4)  Non-FFRDC  work  may  be  done  for  public  sector  and  not-for-profit  entities. 
Commercial  work  shall  not  normally  be  accepted;  however,  should  an  exception  be  granted  by  the 
sponsor,  such  work  must  be  non-proprietary  and  can  not  exclusively  benefit  any  individual  for- 
profit  entity. 

(5)  There  are  no  specified  limits  on  the  volume  of  non-FFRDC  work.  However,  each 
FFRDC  primary  sponsor  should  periodically  assess  whether  the  non-FFRDC  work  performed  by 
the  parent  institution  is  impairing  its  ability  to  perform  its  FFRDC  work. 

(6)  Universities  operating  DoD-sponsored  FFRDCs  are  not  restricted  from  performing 
non-FFRDC  work.  Such  work  must  be  obtained,  however,  in  a  manner  compliant  with  applicable 
procurement  policies  to  ensure  that  work  is  not  acquired  through  an  unfair  advantage  associated 
with  the  FFRDC  mission,  purpose  or  special  relationship. 
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c.  Technology  Transfer  Activities.  Sponsoring  agreements  may  include  authority  for 
Research  and  Development  Laboratory  FFRDCs  to  participate  with  industry  in  technology 
transfer  activities  when  appropriate.  Sponsors  must  include  adequate  safeguards  to  ensure  the 
FFRDC  remains  free  of  organizational  conflicts  of  interest  and  that  the  conditions  for  establishing 
and  maintaining  the  FFRDC  are  not  compromised.  The  safeguards  should  include  specific  review 
and  approval  of  technology  transfer  work  by  the  FFRDC  sponsor  on  a  case-by-case  basis. 


d.  Exceptions.  Requests  for  work  performance  exceptions  (FFRDC  and  non-FFRDC) 
shall  be  directed  to  the  primary  sponsor,  Service  Acquisition  Executive,  or  the  component  head, 
as  applicable. 

2.  FFRDC  Level  of  Effort. 


a.  DoD-funded  Work. 

(1)  Staff  years  of  technical  effort  (STEs)  shall  be  used  in  sizing  and  managing  DoD- 
funded  FFRDC  work.  Although  the  total  number  of  STEs  available  will  be  constrained  by  DoD 
budgetary  considerations  and  statutory  requirements,  STEs  will  provide  a  standard  measure 
across  all  of  DoD’s  FFRDCs  for  projecting  DoD  workload  and  funding  requirements.  Appendix 
B  contains  the  standard  definition  of  STE  to  be  used  in  computing  workload  requirements. 

(2)  The  DDR&E  will  establish  a  workload  ceiling  annually  by  STE  for  each  FFRDC 
based  on:  (a)  sponsor  needs;  (b)  a  determination  that  those  needs  require  one  or  more  of  the  core 
capabilities  of  the  FFRDC;  and  (c)  the  general  guidelines  laid  out  in  subparagraph  2.b.  below. 

(3)  Requests  to  the  DDR&E  for  deviations  from  or  exceptions  to  the  established  STE 
ceiling  for  any  specific  FFRDC  shall  be  presented  by  the  sponsor  with  appropriate  justification. 

b.  General  Guidelines.  Annual  levels  of  effort  shall  be  based  upon  application  of  the  core 
concept  and  the  following  category  guidelines: 

(1)  Studies  and  Analyses  Centers  (S&A) 

(a)  Maintain  a  relatively  stable  level-of-effort. 

(b)  Maintain  competency  in  core  areas. 

(2)  Systems  Engineering  and  Integration  Centers  (SE&I) 

(a)  Maintain  stable  core  competencies. 

(b)  Respond  to  projected  trends  in  workload  and  funding  consistent  with  the 
budget  supporting  the  mission  area. 


(3)  Research  and  Development  (R&D)  Laboratories.  Maintain  technical  expertise  and 
related  competencies  necessary  to  address  the  core  work  and  priorities  of  the  sponsor. 

c.  Non-DoD  Funded  Work.  FFRDC  work  funded  using  non-DoD  appropriations  will 
comply  with  the  same  policies  and  constraints  as  DoD-funded  work  and  will  be  reported  in 
accordance  with  Appendix  D. 

3.  FFRDC  Fee  FFRDCs  are  operated  by  not-for-profit  or  university  affiliated  organizations 
and  are  strictly  limited  by  DoD  in  the  types  of  work  they  may  perform.  By  limiting  the  work  that 
can  be  done  by  a  FFRDC,  the  DoD  limits  the  sources  of  funds  available  to  the  FFRDC  to  pay 
costs  normally  incurred  by  a  business,  but  not  reimbursed  under  Government  cost-type  contracts. 
As  a  result,  fees  for  FFRDC  work  may  be  appropriate.  In  accordance  with  the  OFPP  Policy 
Letter  84-1,  the  appropriateness  of  paying  fees  to  DoD  FFRDCs  should  be  addressed  in  their 
sponsoring  agreements.  When  fees  are  authorized,  the  following  guidelines  are  to  be  used. 

a  The  sponsor/contracting  activity  must  make  a  determination  that  a  fee  is  needed.  This 
determination  is  to  be  performed  annually  by  evaluating  the  FFRDC’s  current  Application  of 
Funds  and  Sources  of  Funds  statements,  and  considering  the  following: 

(1)  Proportion  of  retained  earnings  that  relates  to  the  DoD  contracted  effort. 

(2)  Facilities  capital  acquisition  plans. 

(3)  Working  capital  funding  assessment  based  on  operating  cycle  cash  needs. 

(4)  Funding  of  unreimbursable  costs  deemed  “ordinary  and  necessary”  to  the 
FFRDC’s  continued  successful  operation. 

b.  If  a  fee  is  determined  to  be  needed,  the  sponsor/contracting  activity  will  use  the 
following  as  guidelines  in  arriving  at  the  fee  amount: 

(1)  An  annual  fee  proposal  justifying  each  element  of  fee.  The  proposal  must: 

(a)  Provide  sufficient  visibility  into  each  element  of  fee. 

(b)  Avoid  the  use  of  undefined  and  ambiguous  terms,  such  as  “miscellaneous”  and 

“other.” 


(c)  Not  include  any  cost  for  which  reimbursement,  either  as  an  incurred  cost  or  as 
an  element  of  fee,  is  prohibited  by  statute. 

(d)  Comply  with  fee  reimbursement  restrictions  and/or  limitations  included  in  the 
sponsoring  agreement  and/or  applicable  statutes  and  regulations. 
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(e)  Identify,  for  inclusion  as  an  element  of  fee,  costs  not  reimbursable  under  the 
contract  that  the  FFRDC  can  nevertheless  demonstrate  are  ordinary  and  necessary  to  the 
FFRDC’s  successful  operation. 

(2)  The  extent  to  which  the  prior  representations  and  justifications  regarding  fee  have 
proven  accurate  (both  as  to  the  fee  amount  and  to  the  planned  uses  for  the  fee).  Unexplained  or 
repeated  failure  to  reasonably  adhere  to  planned  uses  for  fee  should  serve  as  a  basis  for 
challenging  either  the  appropriateness  and/or  the  magnitude  of  proposed  fees. 

(3)  Costs  incurred  by  the  FFRDC  that  are  allowable  or  allocable  under  the  cost 
principles  (i.e.,  commercial  using  FAR  31.2,  not-for-profit  using  OMB  Circular  A-122,  or 
university  affiliated  using  OMB  Circular  A-21),  regulations,  or  statutes  applicable  to  that  FFRDC 
should  be  classified  as  direct  or  indirect  (overhead/G&A)  charges  to  the  contract  and  not  included 
as  proposed  fee  elements.  Exceptions  may  be  made  to  this  guideline  with  primary  sponsor 
approval. 

c.  These  guidelines  are  not  intended  to  eliminate  best  practice  techniques  in  applying  the 
applicable  cost  principles,  such  as  implementation  of  an  award  fee  concept 

d.  Appendix  C  elaborates  on  the  process  to  be  used  in  determining  the  need  for  fee. 

E.  RESPONSIBILITIES 

1 .  The  Director  of  Defense  Research  and  Engineering  (DDR&E),  consistent  with  the 
provisions  of  this  plan,  is  responsible  to  the  Deputy  Secretary  of  Defense  through  the  Under 
Secretary  of  Defense  for  Acquisition  and  Technology  to: 

a.  Ensure  that  STEs  established  for  each  of  the  FFRDCs  are  consistent  with  overall  DoD 
requirements  and  strategy. 

b.  Monitor  the  mechanisms  used  by  FFRDC  sponsors  to  ensure  the  appropriateness  and 
value  of  FFRDC  efforts  and  activities. 

c.  Oversee  implementation  and  execution  of  this  management  plan  by  each  FFRDC 
primary  sponsor  to  ensure  compliance. 

2.  The  FFRDC  Primary  Sponsor  shall: 

a.  Ensure  that  each  FFRDC  is  being  used  only  for  its  intended  purposes. 

b.  Ensure  that  the  costs  of  the  goods  and  services  it  provides  are  reasonable,  and  that  it 
produces  high-quality  work  of  value  to  user  organizations. 

c.  Review  descriptions  of  work  proposed  to  be  done  by  the  FFRDC  and  ensure  that  the 
work  assigned  is  consistent  with  the  FFRDC’s  core  statement. 
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d.  Assure  the  DDR&E  that  these  provisions  are  being  satisfied  by  making  a  specific 
statement  in  the  Annual  Review  Assessment  required  in  accordance  with  Appendix  D. 

F.  PROCEDURES 

1  Sponsoring  Agreements.  Primary  sponsors  of  FFRDCs  shall  maintain  sponsoring 
agreements.  The  specific  content  of  these  documents  may  vary  depending  on  the  nature  of  the 
relationship  between  the  primary  sponsor  and  the  FFRDC.  Sponsors  may  supplement  sponsoring 
agreements  with  operating  instructions;  however,  at  a  minimum  sponsoring  agreements  must 
include  the  following: 

a.  Provisions  for  the  orderly  termination  or  nonrenewal  of  the  contract,  disposal  of 
assets,  and  settlement  of  liabilities.  The  responsibility  for  capitalization  of  the  FFRDC  must  be 
defined  in  such  a  manner  that  ownership  of  assets  may  be  readily  and  equitably  determined  upon 
termination  of  the  FFRDCs  relationship  with  its  sponsor. 

b.  A  prohibition  against  the  FFRDC  competing  with  any  non-FFRDC  concern  in 
response  to  a  Federal  agency  formal  Request  for  Proposal  for  other  than  the  operation  of  a 
FFRDC.  This  prohibition  is  not  required  to  be  applied  to  any  parent  organization  in  its  non- 
FFRDC  operations.  However,  sponsoring  agencies  may  expand  this  prohibition  as  they  determine 
necessary  and  appropriate 

c.  A  determination  of  whether  the  FFRDC  may  accept  work  from  other  than  the  primary 
sponsor.  If  non-sponsor  work  can  be  accepted,  a  description  of  the  procedures  to  be  followed 
will  be  included,  along  with  any  limitations  as  to  the  non-sponsors  from  which  work  can  be 
accepted  (e  g.,  other  Federal  agencies.  State,  local  or  foreign  governments,  or  not-for-profit 
organizations). 

d.  A  description  of  the  procedures  used  to  make  an  annual  assessment  to  evaluate 
performance  in  the  areas  of  technical  quality,  responsiveness,  value,  cost  and  timeliness.  Also 
required  is  a  description  of  the  mechanism  used  to  provide  feedback  to  the  FFRDC  in  order  to 
identify  and  resolve  any  perceived  or  real  problems. 

e  When  cost-type  contracts  are  used,  the  sponsor  should  identify  any  cost  elements  or 
fee  that  require  advance  agreement  and/or  approval.  Such  items  may  include,  but  are  not  limited 
to  personnel  compensation,  depreciation,  various  indirect  costs  such  as  Independent  Research  and 
Development,  or  others  as  deemed  appropriate  by  the  sponsor. 

2.  Core  Statement.  Primary  sponsors  of  FFRDCs  shall  maintain  a  current  core  statement 
describing  the  purpose  for  establishing  the  FFRDC,  the  nature  of  the  strategic  relationship 
between  the  FFRDC  and  the  primary  sponsor,  and  a  description  of  its  mission,  general  scope  of 
effort,  and  core  competencies  the  FFRDC  must  maintain  so  that  it  can  assist  in  accomplishing  the 
sponsoring  agency’s  mission.  This  statement  must  be  specific  enough  to  permit  a  discrimination 
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between  work  that  is  within  the  scope  of  effort  for  which  the  FFRDC  was  established  and  work 
that  should  be  performed  elsewhere. 


3 .  Comprehensive  Review.  Prior  to  renewal  of  the  FFRDC  contract,  the  primary  sponsor 
shall  conduct  a  comprehensive  review  of  the  continuing  use  of  and  need  for  the  FFRDC.  This 
review  must  comply  with  Federal  Acquisition  Regulation,  Part  35.01 7.  The  resulting 
determination  to  approve  continuation  or  termination  of  the  sponsorship  shall  be  made  by  the 
head  of  the  sponsoring  agency,  with  the  concurrence  of  the  DDR&E,  prior  to  the  anticipated 
contract  renewal  date.  Also,  the  sponsor  shall  advise  the  DDR&E  upon  the  initiation  of  a 
required  review  and  the  expected  date  of  its  completion.  At  that  time,  the  DDR&E  will  have  the 
opportunity  to  advise  the  sponsor  of  any  special  interest  items  or  requirements  to  be  addressed 
during  the  review.  Appendix  E  contains  guidelines  for  the  conduct  of  comprehensive  reviews. 
Sponsors  should  follow  the  guidelines  to  ensure  consistency  and  thoroughness  in  the  review 
process  within  the  DoD. 

4.  Reports.  DDR&E  requires  specified  and  ad  hoc  reports  in  order  to  comply  with 
Congressional  reporting  requirements  and  to  perform  necessary  oversight  functions  and 
responsibilities.  The  schedule  and  content  of  reports  and  other  submissions  currently  required  are 
shown  in  Appendix  D. 

G.  EFFECTIVE  DATE 


This  DoD  FFRDC  Management  Plan  is  effective  May  1,  1996  and  replaces  the  DoD 
FFRDC  Management  Plan  that  became  effective  on  October  1,  1994. 

APPENDIX  A 

DoD  FEDERALLY  FUNDED  RESEARCH  AND  DEVELOPMENT  CENTERS 


STUDY  AND  ANALYSIS  CENTERS 

CENTER  FOR  NAVAL  ANALYSES,  Alexandria,  VA  SPONSOR:  ASN(RD&A) 

CNA's  work  for  the  Navy,  Marine  Corps,  and  other  sponsors  is  based  on  its  expertise  in 
operations  analysis,  system  requirements  and  acquisition,  resource  analysis,  program  planning, 
and  policy,  strategy  and  doctrine.  Its  integrated  research  program  encompasses  a  broad  range  of 
issues,  including  --  for  example  —  the  development  and  evaluation  of  tactics,  operational  testing  of 
new  systems,  assessment  of  current  capabilities,  logistics  and  readiness,  work-force  management, 
space  and  electronic  warfare,  cost  and  operational  effectiveness  analysis,  assessment  of  advanced 
technology,  force  planning,  and  strategic  implications  of  political-military  developments.  CNA 
performs  such  efforts  for  DoN,  other  DoD  agencies,  and  for  non-DoD  agencies  in  matters 
affecting  the  interests  of  DoN  and  DoD. 
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RAND  PROJECT  AIR  FORCE,  Santa  Monica,  CA 


SPONSOR:  ASAF/A 


Conducts  a  continuous  and  interrelated  program  of  objective  analyses  on  major  cross¬ 
cutting  policy  and  management  issues  of  enduring  concern  to  the  Air  Force,  including  studies  on 
preferred  means  of  developing  and  employing  aerospace  power;  national  security  threats  and 
strategies;  Air  Force  missions,  capabilities,  and  organizations;  strategic  and  tactical  force 
operations;  and  technology,  support,  and  resource  management. 


INSTITUTE  FOR  DEFENSE  ANALYSES  (IDA),  Alexandria,  VA  SPONSOR:  USD(A&T) 

Performs  studies  and  analyses  for  the  Office  of  the  Secretary  of  Defense,  Joint  Staff,  Unified 
Commands  and  Defense  Agencies  in  the  areas  of  defense  systems,  science  and  technology, 
strategy  and  forces,  resource  analysis,  advanced  computing  and  information  processing,  training, 
simulation,  acquisition  process,  and  the  industrial  base.  Provides  analyses  of  test  plans, 
operational  assessments  and  test  results  for  weapons  and  other  systems,  including  new  and 
proposed  equipment  of  all  types. 


RAND  NATIONAL  DEFENSE  RESEARCH  INSTITUTE,  Santa  Monica,  CA 

SPONSOR:  USD(A&T) 

Conducts  a  wide  range  of  research  and  analyses  in  the  areas  of  international  security  and 
economic  policy;  threat  assessment;  defense  strategy  and  force  employment  options;  applied 
science  and  technology;  information  processing  systems;  systems  acquisition,  readiness  and 
support  systems;  and  active-duty  and  reserve  manpower,  personnel,  and  training  for  the  Office  of 
the  Secretary  of  Defense,  Joint  Staff,  Unified  Commands,  and  Defense  Agencies. 


LOGISTICS  MANAGEMENT  INSTITUTE,  McLean,  VA  SPONSOR:  USD(A&T) 

Conducts  studies  and  analyses  for  its  primary  sponsor  the  Office  of  the  Secretary  of 
Defense,  Military  Departments,  Defense  Agencies,  Joint  Staff,  and  Unified  Commands.  Its  core 
mission  areas  are:  material  management,  acquisition,  operational  logistics,  facilities  and 
environment,  and  force  management. 


RAND  ARROYO  CENTER,  Santa  Monica,  CA  SPONSOR:  SARDA 

Conducts  a  wide  range  of  research,  studies  and  analyses  in  the  areas  of  strategy,  force 
design  and  operations;  readiness  and  support  infrastructure;  applied  science  and  technology; 
manpower  and  training;  threat  assessment,  and  Army  doctrine. 
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SYSTEMS  ENGINEERING/INTEGRATION  CENTERS 


AEROSPACE  CORPORATION,  El  Segundo,  CA  SPONSOR:  ASAF/A 

Performs  general  systems  engineering  and  integration  for  space  systems.  Provides  planning, 
systems  definition  and  technical  specification  support;  analyzes  design  and  design  alternatives, 
interoperability,  manufacturing  and  quality  control;  and  assist  with  test  and  evaluation,  launch 
support,  flight  tests,  and  orbital  operations.  Appraises  the  technical  performance  of  contractors. 


MITRE  C3I FFRDC,  Bedford,  MA  and  McLean,  VA  SPONSOR: 

ASD(C3I) 

Performs  general  systems  engineering  and  integration  for  the  Command,  Control, 
Communications,  and  Intelligence  (C3I)  community.  Provides  direct  support  through  program 
definition;  creation  of  plans  and  architectures;  specification  of  technical  requirements;  system 
integration;  analyses  of  design  and  design  alternatives;  integration  of  new  capabilities  into  existing 
systems;  integration  of  multiple  legacy  systems  into  effective  systems  of  systems;  hardware  and 
software  review;  and  test  and  evaluation.  Appraises  contractors'  technical  performance. 


RESEARCH  AND  DEVELOPMENT  LABORATORIES 

SOFTWARE  ENGINEERING  INSTITUTE,  Pittsburgh,  PA  SPONSOR:  DARPA 

SEI  is  charged  with  bringing  technology  to  bear  on  rapid  improvement  of  the  quality  of 
operational  software  in  software  intensive  systems;  with  accelerating  the  reduction  to  practice  of 
modem  software  engineering  technology  and  promulgating  the  use  of  this  technology  throughout 
the  software  community;  and  with  fostering  standards  of  excellence  for  improving  software 
engineering  practice. 


MIT  LINCOLN  LABORATORY,  Lexington,  MA  SPONSOR:  ASAF/A 

The  laboratory  carries  out  a  program  of  research  and  development  in  a  number  of 
technologies.  Program  activities  extend  from  fundamental  investigations  through  design, 
development,  and  field  test  of  prototype  systems  using  new  technologies. 


IDA  COMMUNICATIONS  AND  COMPUTING  LABORATORY 

Bowie,  MD;  Princeton,  NJ,  LaJolla,  CA  SPONSOR:  NSA 
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Conducts  fundamental  research  for  the  NSA  in  (1)  cryptology,  including  the  creation  and 
analysis  of  complex  encipherment  algorithms,  as  well  as  in  speech  and  signal  analyses;  and  (2) 
various  technologies  associated  with  supercomputing  and  parallel  processing  including  new 
architectures,  hardware,  and  software  (including  prototypes),  as  well  as  parallel  processing 

algorithms  and  applications. 
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APPENDIX  B 


STAFF  YEAR  OF  TECHNICAL  EFFORT  (STE) 


In  calculating  workload  requirements  to  be  delivered  during  the  fiscal  year,  FFRDCs  and 
primary  sponsors  shall  use  the  standard  definition  of  STE  and  work  year  shown  below. 

STEs  apply  to  direct  professional  and  consultant  labor,  performed  by  researchers, 
mathematicians,  programmers,  analysts,  economists,  scientists,  engineers,  and  others  who  perform 
professional-level  technical  work  primarily  in  the  fields  of  studies  and  analyses;  system 
engineering  and  integration;  systems  planning;  program  and  policy  planning  and  analysis;  and 
basic  and  applied  research. 

Minimum  educational  requirements  for  STE  employees  and  consultants  are  a  baccalaureate 
degree  from  an  accredited  college  or  university.  In  rare  instances,  non-degree  personnel  may  be 
included,  but  only  if  they  possess  the  equivalent  of  a  baccalaureate  degree  in  education  and 
experience,  and  are  performing  work  of  the  same  type  and  level  as  that  performed  by  degreed 
STE  employees. 

A  STE  work  year  is  defined  to  be  1,810  hours  of  paid  effort  for  technical  services.  STE 
work  years  include  both  FFRDC  employee  and  subcontracted  consultant  technical  effort. 
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APPENDIX  C 


DETERMINATION  OF  FFRDC  NEED  FOR  FEE 

When  the  sponsor  and  contracting  activity  perform  the  “need”  evaluation,  all  elements  of  the 
FFRDC’s  expenses  (Application  of  Funds)  should  be  analyzed  and  then  compared  to  the 
projected  sources  of  income  (Sources  of  Funds)  As  an  example,  if  income  exceeded  expenses  for 
the  prior  fiscal  year,  there  would  be  a  surplus  to  working  capital  which  should  be  considered  in 
the  following  year’s  determination  of  fee  needs.  Such  surplus  should  be  evaluated  in  light  of  the 
FFRDC’s  working  capital  needs  and  its  reasonable  needs  for  resources  to  apply  to  other  uses, 
such  as  debt  retirement  or  compliance  with  financial  accounting  standards.  If,  on  the  other  hand, 
expenses  exceeded  income  for  the  prior  year  or  are  forecasted  to  do  so  for  the  current  year,  there 
may  be  a  need  to  replenish  the  FFRDC’s  available  financial  resources  in  order  to  allow  it  to 
continue  efficient  operations. 

The  sponsor’s  and  contracting  activity’s  recognition  of  the  need  for  a  fee  should  also 
consider  the  benefit  provided  to  the  operation  and  purpose  of  the  FFRDC.  Activities  that  benefit 
a  parent  institution  as  a  whole  (for  example,  use  of  fee  to  provide  working  capital  to  meet  the 
payroll)  may  be  appropriate  if  there  is  a  benefit  to  DoD.  Conversely,  activities  whose  primary 
purpose  is  to  benefit  or  enhance  a  non-FFRDC  corporate  parent  or  affiliate,  or  to  expand  the 
corporation’s  work  for  sponsors  other  than  the  DoD,  shall  not  serve  as  justification  for  needing  a 
fee  or  be  used  in  establishing  the  fee  amount. 

The  examples  of  cost  categories  that  may  be  used  to  justify  fees  and  establish  fee  amounts 

follow: 


1.  Working  Capital:  Fee  may  reflect  the  amount  of  funds  necessary  to  fund  the  normal 
business  operations  of  the  FFRDC,  as  assessed  on  an  operating  cycle  basis.  Specifically,  fee  may 
reflect  the  working  capital  needs  of  the  FFRDC.  Working  capital  represents  funds  available  to  the 
FFRDC  to  pay  current  operating  expenses  (between  the  time  the  cost  is  incurred  and 
reimbursement  is  received).  The  FFRDC s  may  either  use  their  own  reserves  (to  the  extent  such 
reserves  are  in  liquid  form)  or  borrow,  thereby  incurring  interest  expense,  to  satisfy  the  FFRDC’s 
working  capital  needs. 

2.  Facilities  Capital:  Fee  may  reflect  the  costs  of  fixed  asset  acquisitions  in  accordance 
with  capital  acquisition  plans  that  are  approved  by  the  Government  as  a  part  of  the  annual  Fee 
Justification  Proposal.  Care  should  be  taken  to  ensure  that  the  planned  capital  purchases  do  not 
include  items  that  must  be  charged  to  the  contract  in  their  entirety  at  the  time  of  acquisition  under 
the  cost  principles  to  which  the  FFRDC  is  subject.  However,  FFRDCs  need  to  be  able  to  acquire 
the  tangible  assets  necessary  for  the  effective  and  efficient  conduct  of  their  operations. 

Government  and  cost  accounting  regulations  and  the  tax  code  may  require  that  the  cost  of  these 
assets  be  capitalized  and  recovered  through  depreciation  or  amortization  over  a  period  of  years, 
even  though  the  financial  resources  used  to  acquire  them  have  to  be  committed  at  the  time  of 
acquisition.  Such  capital  acquisitions  justify  fee  to  the  extent  of  both  the  timing  differences  and 
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the  need  to  service  and  retire  debt  that  may  have  been  incurred  in  the  original  acquisition 
transaction.  Conversely,  when  feasible,  capital  equipment  and  real  and  leasehold  improvements 
should  not  exceed  the  depreciation/amortization  (equipment  and  building)  and  cost  of  money 
income. 

3.  Other  Unreimbursed  Expense:  Fee  may  reflect  costs  that  will  be  used  by  the  FFRDC 
to  pay  for  a  variety  of  other  expenses  not  included  in  the  above  two  categories.  These  types  of 
expenses  must  be  “ordinary  and  necessary”  to  the  operation  of  the  FFRDC  and  should  not  include 
allowable  or  allocable  costs  (direct  or  indirect)  that  can  be  charged  to  the  contract.  Fee  serves  as 
the  only  source  of  funds  to  pay  such  expenses.  However,  if  there  is  sufficient  justification  for 
including  additional  expenses  in  fee,  that  can  be  permitted  if  justified  to  the  satisfaction  of  the 
sponsor.  In  order  for  these  expenses  to  become  appropriate  for  consideration  as  a  fee  need,  they 
must  be  separately  identified  and  justified  in  the  annual  fee  proposal.  The  projected  occurrence  of 
such  expenses  does  not  in  itself  justify  the  allowance  of  fee;  i.e.,  the  FFRDC  must  establish  that 
the  expenses  are  “ordinary  and  necessary”  for  its  successful  operation. 
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APPENDIX  D 


REPORTING  REQUIREMENTS  FOR  FFRDC  PRIMARY  SPONSORS 


ANNUAL  REPORTING 
REQUIREMENTS 

DUE  DATE 

DESCRIPTION 

Annual  Report  on  Staff 

Years  of  Technical  Effort 
(STE)  and  Funding. 

15  November 

Provide  DDR&E  with  a  report  showing 
STEs  and  associated  funding  data  (DoD 
and  non-DoD).  DDR&E  will  provide 
required  data  call  format  necessary  for: 

(1)  Congressional  Reporting 

(2)  Budget  Estimates. 

Mid-Year  Status  Update 

30  April 

Provide  DDR&E  a  report  for  use  in 
monitoring  FFRDC  obligations  (DoD  and 
non-DoD)  per  the  data  call  format.  The 
report  should  address  the  sponsor’s  ability 
to  use  and  fund  all  authorized  DoD- 
funded  STEs;  if  they  anticipate  having 
excess  STEs  available;  and  if  they 
anticipate  submission  of  request(s)  for 
exception(s). 

Annual  Review  Assessment 

30  days  after  completion 
of  the  assessment 

Provide  to  the  DDR&E  a  copy  of  the 
annual  review  assessment.  The 
requirements  for  an  annual  assessment 
may  be  met  by  the  Comprehensive 

Review  during  the  year  that  a 
comprehensive  review  is  required. 

Changes  to  Sponsoring 
Agreement,  Core  Statement 

Within  30  days  of 
change  implementation 

Provide  the  DDR&E  with  copies  of 
changes  to  the  Sponsoring  Agreement  or 
Core  Statement. 

Comprehensive  Review 
Notification 

One  year  prior  to  Due 
Date  of  the  Review 

Advise  the  DDR&E  of  Comprehensive 
Review  initiation.  DDR&E  will  advise 
the  sponsor  of  any  special  review 
requirements. 

Comprehensive  Review 

NLT  90  days  prior  to 
renewal  of  the  FFRDC 
contract 

Provide  to  the  DDR&E  the  results  of  the 
Comprehensive  Review  for  the  use  and 
need  of  the  FFRDC  in  accordance  with 
this  Plan  (see  Appendix  E),  OFPP  Policy 
Letter  84-1,  and  FAR  Part  35.017. 

DDR&E  concurrence  is  required  prior  to 
renewal  of  the  FFRDC  contract. 

APPENDIX  E 


COMPREHENSIVE  REVIEW  GUIDELINES  FOR 
DoD-SPONSORED 

FEDERALLY  FUNDED  RESEARCH  AND  DEVELOPMENT  CENTERS 


PURPOSE:  The  purpose  of  the  comprehensive  review  is  to  formally  analyze  the  use  and  need  for 
the  FFRDC  in  order  to  assist  the  head  of  the  sponsoring  agency  in  determining  whether  to  continue 
sponsorship  of  the  FFRDC. 

This  appendix  provides  the  guidelines  for  reporting  the  results  of  FFRDC  comprehensive 
reviews  in  accordance  w'ith  this  management  plan,  OFPP  Policy  Letter  84-1,  and  the  FAR. 

-  Identify  the  FFRDC,  its  primary  sponsor  and  contracting  activity.  Include  the  date  and  term  of 
the  FFRDC's  current  sponsoring  agreement. 

Provide  a  detailed  examination  of  the  sponsor's  special  technical  needs  and  mission 
requirements  that  are  being  performed  by  the  FFRDC  to  determine  whether,  and  at  what  level, 
they  should  continue  to  exist  (FAR  35.017-4  (c)(1)). 

Identify  requirements  for  FFRDC  support  including  known  specific  programs  involved, 
the  level  of  effort  required  and  the  types  of  tasks  to  be  performed. 

Consideration  of  alternative  sources  (FAR  35.1 07-4(c)(2)) : 

Specify  the  special  research,  systems  development,  or  analytical  needs,  skills,  and/or 
capabilities  involved  in  accomplishing  FFRDC  tasks. 

Explain  why  the  capabilities  cannot  be  provided  as  effectively  by  in-house  personnel, 
for-profit  or  not-for-profit  contractors,  university-affiliated  organizations,  or  another 
existing  FFRDC.  Include  statements  on  the  alternatives  to  the  FFRDC  that  were 
considered  and  the  rationale  for  not  selecting  each  of  them. 

Provide  a  detailed  assessment  of  the  efficiency  and  effectiveness  of  the  FFRDC  in  meeting  a 
sponsor's/user's  needs  including  the  FFRDC's  ability  to  maintain  its  objectivity,  independence, 
quick  response  capability,  currency  in  its  field(s)  of  expertise,  and  familiarity  with  the  needs  of 
its  sponsor  (FAR  35.01 7-4(c)(3)). 

Include  a  summary  of  FFRDC  accomplishments  and  their  effectiveness  in  meeting  user 
needs  since  the  last  comprehensive  review.  As  a  minimum,  the  quality  and  timeliness 
of  the  work  produced,  the  number  and  dollar  value  of  projects  and  programs  assessed, 
and  the  user  evaluations  of  performance  should  be  addressed.  A  summary  of  the  results 
of  the  most  recent  annual  review  should  be  included.  All  major  users  should  participate 
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in  this  portion  of  the  comprehensive  review.  Discuss  any  criticisms  or  concerns  that  the 
users  had  with  FFRDC  performance  and  the  steps  taken  to  resolve  those  issues. 

-  Conduct  an  assessment  of  the  FFRDC  management  controls  to  ensure  cost-effective  operation 
(FAR  35.017-4(c)(4)). 

Discuss  accounting  and  purchasing  systems;  overhead  costs  and  fees;  oversight  actions 
taken  to  verify  cost-effective  operations;  and  other  management  issues  as  deemed 
appropriate. 

-  Provide  a  determination  that  the  criteria  for  establishing  the  FFRDC  are  satisfied  and  that  the 
sponsoring  agreement  is  in  compliance  with  FAR  35.017,  FAR  35.017-2,  and  the  DoD 
Management  Plan.  Include  a  statement  addressing  each  of  the  criteria.  Provide  a  certification 
that  the  current  sponsoring  agreement  accurately  reflects  the  mission  of  the  FFRDC. 

Discuss  agreements  between  the  Government  and  the  FFRDC.  These  agreements  may 
cover  such  items  as  authorization  of  fees,  provision  of  Government  facilities  and 
equipment,  distribution  of  residual  assets  of  settlement  and  liabilities  in  event  of 
dissolution,  maintenance  of  specific  cash  reserves,  and  waivers  to  accounting  policies  or 
regulatory  requirements. 

-  The  comprehensive  review  should  provide  a  recommended  course  of  action  and  be  signed  by 
the  head  of  the  sponsoring  agency.  DDR&E  concurrence  with  the  results  of  the 
comprehensive  review  is  required  prior  to  renewal  of  the  contract  or  termination  of  the 
FFRDC. 
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DoD  FFRDC  MANAGEMENT  PLAN  AND 
REPORT  ON  THE  DEPARTMENT  OF  DEFENSE 
FIVE-YEAR  PLAN 
FOR 

FEDERALLY  FUNDED  RESEARCH  AND  DEVELOPMENT  CENTERS  (FFRDCs) 

AND 

UNIVERSITY  AFFILIATED  RESEARCH  CENTERS  (UARCs) 


Subsection  220(f)  of  the  National  Defense  Authorization  Act  for  Fiscal  Year  1996  directs  the 
Department  to  submit  a  Five-Year  Plan  to  establish  a  framework  for  the  future  workload  of  each 
of  its  Federally  Funded  Research  and  Development  Centers  (FFRDCs)  and  University  Affiliated 
Research  Centers  (UARCs).  The  plan  is  required  to  address: 

•  Reduction  and  consolidation  of  the  activities  performed  by  FFRDCs  and  UARCs,  and  a 
framework  for  the  future  workload  of  such  centers; 

•  Implementation  by  FFRDCs  and  UARCs  of  only  core  activities  that  require  the  unique 
capabilities  and  arrangements  afforded  by  such  centers  (by  October  1,  2000);  and, 

•  An  assessment  of  the  number  of  staff  years  of  effort  needed  in  each  FFRDC  and  UARC 
over  the  next  five  years. 


Background 

FFRDCs  are  not-for-profit,  private  sector  organizations  that  maintain  long-term,  strategic 
relationships  with  the  Department  of  Defense  (DoD)  and  operate  in  the  public  interest,  free  from 
real  or  perceived  conflicts  of  interest.  FFRDCs  maintain  specific  expertise  in  areas  critical  to  DoD 
and  perform  research,  development  and  analytical  tasks  integral  to  the  mission  and  operations  of 
sponsoring  DoD  components. 

Similarly,  UARCs  are  not-for-profit,  private  sector  organizations  affiliated  with,  or  part  of, 
universities  or  colleges  that  maintain  essential  research,  development  and  engineering  capabilities 
needed  by  sponsoring  DoD  components.  The  UARCs  are  a  recent  DoD  designation  of  university 
affiliated  laboratories  doing  over  $2  million  of  core  work  annually  for  DoD  using  management 
guidelines  tailored  to  their  unique  structure.  They  too  maintain  long-term,  strategic  relationships 
with  sponsoring  DoD  components  in  specific  core  areas  and  operate  in  the  public  interest,  free 
from  real  or  perceived  conflicts  of  interest.  FFRDCs  and  UARCs  are  financed  through  long-term, 
non-competitive  contracts  awarded  by  sponsoring  DoD  components  for  specified  core  work. 

Over  the  past  year,  the  Department  has  carefully  reviewed  its  relationships  with  FFRDCs  and 
UARCs.  The  Under  Secretary  of  Defense  for  Acquisition  and  Technology  (USD(A&T))  formed 
a  senior  level  DoD  Advisory  Group,  chaired  by  the  Director,  Defense  Research  and  Engineering 
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(DDR&E),  with  senior  representation  from  OSD,  Joint  Staff,  military  departments,  and  DoD 
agencies,  to  examine  the  issues.  Also,  USD(A&T)  chartered  an  independent  review  of  the  role  of 
DoD  FFRDCs  by  the  Defense  Science  Board.  Both  groups  concluded  that  DoD  still  needs  these 
organizations.  The  Defense  Science  Board  stated  “...the  FFRDCs  should  be  retained  on  the 
strength  of  their  quality  and  the  special  relationships  they  have  with  their  sponsors  on  matters 
which  are  of  great  importance  to  the  Department  of  Defense.”  The  internal  Advisory  Group 
reached  a  similar  conclusion  after  reviewing  alternatives  to  FFRDCs  and  UARCs.  The  bottom 
line  is  that  DoD  believes  that  FFRDCs  and  UARCs  are  doing  high-quality,  high-value  technical 
and  analytic  work  that  could  not  be  provided  as  effectively  by  other  means. 

The  DoD  has  taken  actions  to  deal  with  concerns,  both  real  and  perceived,  that  the  FFRDCs  and 
UARCs  have  not  been  right-sized;  that  they  are  working  in  areas  beyond  the  core  interests  of  the 
sponsoring  DoD  components;  and  that  FFRDCs  and  UARCs  are  using  their  unique  status  to  gain 
unfair  competitive  advantage  over  commercial  entities.  Four  major  initiatives  have  been 
introduced  to  improve  the  effectiveness  of  our  management: 

1 .  Limiting  the  program  content  of  FFRDCs  and  UARCs  to  “core”  work; 

2.  Establishing  stringent  criteria  for  the  acceptance  of  non-core  work  by  a  FFRDC’s  or 
U ARC’S  parent  institution; 

3.  Chartering  an  independent  advisory  committee  to  review  DoD’s  management  and 
oversight  of  FFRDCs  and  UARCs;  and, 

4.  Developing  a  new  set  of  guidelines  to  ensure  that  fees  provided  to  FFRDCs  are  based 
on  justified  need. 

The  DoD  believes  these  initiatives,  along  with  the  support  of  the  Congress,  will  effectively 
address  concerns  about  FFRDC  and  UARC  management  and  will  facilitate  the  continued  use  of 
the  critical  capabilities  provided  by  these  centers.  As  the  Department  downsizes,  the  FFRDCs  and 
UARCs  have  become  increasingly  important  as  centers  of  independent  technical  expertise  and 
analytical  support. 

Reduction  and  Consolidation  of  FFRDCs  and  UARCs 

DoD  currently  sponsors  1 1  FFRDCs  and  six  UARCs  that  collectively  account  for  about  four 
percent  of  DoD’s  Fiscal  Year  (FY)  1996  Research,  Development,  Test  and  Evaluation  (RDT&E) 
budget.  The  DoD  Advisory  Group  considered  and  evaluated  alternatives  to  FFRDCs  (including 
eliminating  and  consolidating)  in  terms  of  risk,  quality  and  cost.  The  Group  determined  the 
current  number  of  FFRDCs  appropriate  for  existing  and  anticipated  Departmental  needs  over  the 
next  five  years. 

The  Group  also  determined  that  reducing  the  number  of  FFRDCs  through  consolidation  would 
not  result  in  significant  cost  savings.  On  the  other  hand,  consolidation  would  risk  the  loss  of 
focus  on  the  needs  of  FFRDC  sponsors.  The  expertise  each  FFRDC  offers  the  DoD  is  unique  to 
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the  needs  of  the  sponsors  and  does  not  represent  duplication  of  effort  that  can  be  effectively 
eliminated  by  consolidation.  Each  of  the  FFRDCs  fall  into  one  of  three  categories  with  differing 
arguments  against  consolidation. 

•  Studies  and  Analyses  (S&A")  Centers:  There  are  six  DoD-sponsored  S&A  centers.  They 
deliver  independent  and  objective  analyses,  and  advise  their  sponsors  in  individually  unique 
core  areas  important  for  policy  development,  decision  making,  alternative  approaches,  and 
new  ideas  on  issues  of  significance.  The  amount  of  funding  spent  on  these  centers 
represents  only  about  18  percent  of  the  total  funding  for  FFRDCs  in  FY1996. 

•  Systems  Engineering  and  Integration  (SE&D  Centers:  DoD  sponsors  two  SE&I  centers. 
They  provide  required  support  in  core  areas  not  available  from  sponsor  s  in-house 
technical  and  engineering  capabilities  to  ensure  that  complex  systems  will  meet  operational 
requirements.  One,  the  Aerospace  Corporation,  specializes  in  general  systems  engineering 
and  integration  for  space  systems.  The  other,  MITRE  C3I FFRDC,  performs  general 
systems  engineering  and  integration  for  the  command,  control,  communications,  and 
intelligence  (C3I)  community.  The  mission,  purpose  and  core  capabilities  of  the  two 
centers  are  completely  different.  Consolidation  is  not  an  option. 

•  Research  &  Development  (R&D)  Laboratories:  R&D  laboratories  fill  voids  in  different 
technology  areas  where  in-house  and  private  sector  research  and  development  centers  are 
unable  to  meet  DoD  core  area  needs.  Specific  objectives  for  these  FFRDCs  are  to:  (1) 
maintain  over  the  long-term  a  competency  in  technology  areas  where  the  Government 
cannot  rely  on  in-house  or  private  sector  capabilities;  and  (2)  develop  and  transfer 
important  new  technology  to  the  private  sector  so  the  Government  can  benefit  from  a 
wider,  broader  base  of  expertise.  R&D  laboratories  engage  in  research  programs  that 
emphasize  the  evolution  and  demonstration  of  advanced  concepts  and  technology,  and  the 
transfer  or  transition  of  technology. 

The  six  UARCs  are  university  research  laboratories  that  perform  basic  research,  design  and 
development  activities  in  support  of  their  DoD  sponsors’  missions.  Each  UARC  fills  a  void  in  a 
different  technology  area  where  in-house  and  private  sector  research  and  development  centers  are 
unable  to  meet  priority  DoD  core  area  needs.  Like  the  R&D  laboratories,  UARCs  maintain  long¬ 
term  competencies  in  particular  core  areas  and  develop  and  transfer  important  new  technology  to 
the  private  sector.  The  Department  has  determined  the  six  UARCs  fulfill  essential  requirements 
for  their  sponsors  without  overlap. 

Significant  reductions  in  the  size  and  budgets  of  the  FFRDCs  were  made  between  FY1991  and 
FY1 996.  As  a  result  of  actions  taken  by  DoD  and  annual  cuts  made  by  the  Congress  in  the 
budget,  the  total  funding  for  FFRDCs  has  decreased  about  34  percent  since  FY1991 .  The  overall 
downward  adjustment  to  the  RDT&E  budget  during  the  same  period  was  about  17  percent.  The 
DoD  has  examined  the  current  and  projected  size  and  budget  of  each  FFRDC  and  UARC  to 
determine  whether  they  are  appropriate  to  meet  the  primary  needs  of  the  sponsors  and  ensure  the 
FFRDCs  and  UARCs  maintain  the  core  competencies  necessary  to  meet  those  needs.  The 
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Department  is  concerned  that  further  reductions  could  force  these  institutions  below  the  level 
needed  to  maintain  critical  core  competencies. 

FFRDC  and  UARC  Performance  of  Core  Activities 


The  Department,  in  furthering  its  proactive  management  approach,  implemented  a  “core”  work 
concept  in  FY1995  for  managing  the  workload  of  the  FFRDCs  and  UARCs.  The  “core”  concept 
is  a  way  to  ensure  FFRDCs  and  UARCs  are  doing  the  kinds  of  work  they  should  be  doing  and  are 
right-sized.  The  “core”  concept  has  also  refined  the  way  workload  is  projected  and  assessed,  and 
has  become  the  focus  of  the  revised  DoD  FFRDC  Management  Plan. 

Core  Statement.  For  each  FFRDC  and  UARC,  the  sponsoring  DoD  component  developed  a 
statement  defining  core  work.  The  core  statement  is  based  on  the  mission,  purpose,  core 
competencies,  and  the  strategic  relationship  that  exists  between  that  sponsor  and  the  FFRDC  or 
UARC.  It  is  a  dynamic  document,  periodically  revised,  to  reflect  evolving  core  requirements  of 
the  sponsoring  DoD  component.  The  core  statements  developed  by  the  DoD  sponsors  (in 
FY1995)  were  initially  subject  to  USD(A&T)  approval,  and  any  changes  to  the  approved  core 
statements  must  be  submitted  to  DDR&E  for  concurrence  before  they  can  become  effective. 

In  addition,  each  DoD  sponsor  developed  and  applied  specific  core  criteria  to  determine  whether 
a  task  is  within  the  scope  of  the  core  statement.  These  criteria  were  applied  to  all  ongoing 
FY1995  work  and  to  each  proposed  task  submitted  for  FY1996.  As  a  result  of  the  program 
assessment,  sponsors  identified  a  total  of  about  $43  million  as  non-core  work  in  the  FFRDCs  and 
about  $26  million  in  the  UARCs.  Many  of  these  ongoing  non-core  tasks  have  been  transitioned 
out  of  the  FFRDCs  and  UARCs.  Those  remaining  will  be  transitioned  out  by  January  31,  1997. 

The  minimum  criteria  established  by  DDR&E  for  determining  FFRDC/UARC  “core”  work  are 
shown  below. 

1.  Consistency  with  the  center’s  mission,  purpose  and  capabilities.  The  work 
assigned  must  be  consistent  with  the  center’s  basic  mission  and  purpose,  must  be  sponsored  by  an 
appropriate  organization,  and  must  be  performed  effectively  using  the  capabilities  that  the  center 
maintains. 

2.  Consistency  with  the  center’s  core  competencies  The  work  must  fall  within  the 
center’s  core  competencies— as  specifically  defined  in  its  core  statement. 

3.  Consistency  with  the  center’s  strategic  relationship  The  work  must  be 
appropriate  for  an  organization  maintaining  a  strategic  relationship  with  DoD,  as  evidenced  by  the 
need  for  one  or  more  of  the  following: 

a  Effective  performance  of  objective,  high-quality  work  on  subjects  integral  to  the 
mission  and  operations  of  sponsoring  offices. 

b.  Freedom  from  real  and  perceived  conflicts  of  interest  caused  by  Service, 
commercial,  or  other  involvement. 
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c.  Broad  access  to  information,  including  sensitive  Government  information, 
proprietary  data  from  industry,  and  other  information  not  normally  available  outside  the 
Government. 

d.  Comprehensive  knowledge  of  sponsor  needs,  problems  and  issues. 

e.  Responsiveness  to  emerging  and  evolving  needs  of  sponsors. 

f.  Long-term  continuity  of  knowledge  on  issues  and  problems  of  enduring  concern, 
including  both  maintaining  corporate  memory  when  appropriate  and  responding  to  quick-response 
sponsor  needs  in  areas  of  established  expertise. 

Core  Workload.  Given  the  mission  of  the  FFRDCs  and  UARCs,  staff  years  of  technical  effort 
(STEs)  are  the  best  measure  for  core  workload.  STEs  apply  to  direct  professional  and  consultant 
labor,  and  others  who  perform  professional-level  technical  work.  A  STE  work  year  is  defined  to 
be  1,810  hours  of  full  time  employee  or  consultant  effort,  including  subcontracted  efforts  for 
technical  services. 

For  FFRDCs,  DDR&E  will  annually  determine  how  many  STEs  are  required  by  each  FFRDC 
based  on  several  factors,  including  sponsor  needs,  special  circumstances,  management  strategy 
(e.g.,  core  concept,  acquisition  reform  initiatives,  budget  tracks),  statutory  direction,  and  the 
guidelines  for  determining  workload  for  each  category  of  FFRDC.  The  guidelines,  to  be  applied 
by  the  FFRDC  sponsor  in  projecting  workload  and  funding  requirements,  for  each  category  are 
shown  below. 

•  Studies  and  analyses  (S&A1  centers  shall  maintain  a  relatively  stable  annual  level-of-effort 
in  order  to  maintain  competency  in  their  core  areas.  It  is  critical  that  S&A  centers  avoid 
the  loss  of  continuity  and  expertise  that  arises  from  major  changes  in  staff  levels. 

•  Systems  engineering  and  integration  ( SE&I)  centers  shall  maintain  stable  core 
competencies  and  respond  to  projected  trends  in  workload  and  funding  consistent  with  the 
budget  supporting  the  FFRDC  mission  area. 

•  Research  and  development  (R&Dt  centers  shall  maintain  technical  expertise  and  related 
competencies  necessary  to  address  core  work  and  priorities  of  the  sponsor. 

The  process  for  UARCs  is  similar.  The  focus  is  on  ensuring  that  annual  STEs  at  each  UARC 
represent  those  essential  engineering,  research,  and  development  capabilities  defined  in  their  core 
statement. 

From  the  annual  workload  requirements  provided  by  the  sponsors,  the  DDR&E  will  approve  STE 
levels,  consistent  with  overall  DoD  requirements  and  strategy,  and  maintain  a  five-year  projection 
for  planning  purposes.  Further,  the  DDR&E  will  monitor  sponsor  execution  against  approved 
STE  levels  and  budget  projections  based  on  STE  levels  during  the  fiscal  year  and  take  action  as 
required.  Requests  to  the  DDR&E  for  deviations  from,  or  exceptions  to,  the  established  STE 
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level  for  any  specific  FFRDC  or  UARC  shall  be  presented  by  the  FFRDC  or  UARC  sponsor  with 
appropriate  justification. 

Management  Plans.  A  revised  DoD  FFRDC  Management  Plan,  to  become  effective  on  May  1 , 
1996,  supports  the  DoD’s  major  initiatives  by  establishing  the  policies  and  procedures  described 
above  for  the  management  and  use  of  DoD-sponsored  FFRDCs.  It  applies  to  FFRDC  primary 
sponsors,  FFRDC  users,  and  contracting  activities  that  award  FFRDC  contracts.  A  copy  of  the 
DoD  FFRDC  Management  Plan  is  attached  (Attachment  1).  Also  effective  May  1,  1996  is  the 
DoD  UARC  Management  Plan  (Attachment  2). 

Five-Year  Assessment  of  FFRDC  and  UARC  Requirements  for  STE 

As  a  result  of  applying  the  “core”  concept,  the  sponsors  of  the  DoD  FFRDCs  and  UARCs,  and 
the  DDR&E,  developed  five-year  projections  of  the  STE  required  as  shown  below. 
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FFRDCs 

FY1996 

STE 

(Staff 

Years) 

FY  1997 
STE 
(Staff 
Years) 

FY  1998 
STE 
(Staff 
Years) 

FY  1999 
STE 
(Staff 
Years) 

FY  2000 
STE 
(Staff 
Years) 

RAND-NDRI 

103.9 

104.0 

106.0 

106.0 

106.0 

RAND- Arroyo 

96.8 

100.0 

104.0 

104.0 

104.0 

RAND-Project  Air  Force 

113.0 

113.0 

113.0 

113.0 

113.0 

Center  for  Naval  Analyses 

233.5 

245.0 

252.0 

259.0 

266.0 

Logistics  Management  Institute 

148.0 

156.0 

156.0 

156.0 

156.0 

Institute  for  Defense  Analyses-S&A 

286.0 

300.0 

305.0 

310.0 

315.0 

Institute  for  Defense  Analyses-OT&E 
(Note  1) 

66.5 

70.0 

68.9 

68.0 

67.3 

Studies  and  Analysis  Subtotal 

1047.7 

1088.0 

1104.9 

1116.0 

1127.3 

MITRE  C-'l  (Note  2) 

■BS& 

■BUS 

1895.0" 

/  Ifcg  X 

Aerospace 

1765.0 

■BESS! 

SE&I  Subtotals 

3660.0 

3660.0 

3660.0 

|  3660.0 

|  3660.0 

MIT-Lincoln  Laboratory  (Note  3) 

920.0 

920.0 

920.0 

920.0 

920.0 

Institute  for  Defense  Analyses-C3l 

133.5 

144.0 

150.0 

150.0 

Software  Engineering  Institute 

155.3 

163.0 

171.2 

179.7 

188.7 

Laboratory  Subtotal 

1208.8 

1227.0 

1241.2 

1249.7 

1258.7 

FFRDC  Subtotal 

5916.5 

5975.0 

6006.1 

6025. 7|  6046.0 

UARCs 

Johns  Hopkins  University-APL 

1226.0 

1177.0 

1130.0 

1084.0 

1041.0 

Pennsylvania  State  Univ.  ARL 

306.0 

305.0 

303.0 

301.0 

300.0 

University  of  Texas-ARL 

345.3 

335.0 

324.9 

315.2 

305.7 

University  of  Washington-APL 

59.2 

57.4 

55.7 

54.0 

52.4 

Georgia  Tech  Research  Institute 

75.0 

71.0 

64.0 

65.0 

60.0 

Utah  State  University-SDL  (Note  4) 

120.0 

85.0 

25.0 

0.0 

0.0 

UARC  Subtotal 

2131^5 

2030.4 

1902.6 

1819.2 

1759.1 

FFRDC/UARC  Total 

8048.0  8005.4 

7908.7 

7844.9 

7805.1 

Note  1 :  IDA’s  S&A  and  OT&E  activities  are  managed  as  a  single  FFRDC  under  a  common  contract  and  sponsoring  agreement. 

Note  2:  FY1 996  includes  30  STE  of  "core”  work  for  the  National  Photo  Interpretation  Center.  In  FY1 997,  funding  for  these  STE  will  be 
assumed  by  DoD’s  National  Imaging  and  Mapping  Agency. 

Note  3:  STE  level  does  not  include  Laboratory  technical  subcontracts  with  U.S.  research-related  industry  in  support  of  such  Laboratory 
work  field  demonstrations,  test  systems,  and  new  sensor  systems. 

Note  4:  BMDO’s  sponsorship  of  USU/SDL  will  diminish  as  the  Midcourse  Space  Experiment  (MSX)  effort  ends  under  the  current 
contracts  during  FY98.  Other  efforts  funded  through  FY97  will,  if  continued,  be  through  a  new  contract  citing  a  different  authority. 
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Summary 


The  DoD  has  developed  a  Five-Year  Plan  to  establish  a  framework  for  the  future  workload  of 
DoD-sponsored  FFRDCs  and  UARCs.  The  plan  reflects  the  proactive  approach  taken  by 
USD(A&T)  and  DDR&E,  as  well  as  the  valuable  advice  of  an  internal  DoD  Advisory  Group  and 
the  Defense  Science  Board  regarding  DoD’s  management,  use  and  oversight  of  FFRDCs  and 
UARCs.  The  plan  also  emphasizes  DoD’s  management  initiatives  to  implement  the  core  concept, 
in  terms  of  defining  core  work  and  guidelines  for  core  workload,  and  the  policies  and  procedures 
developed  to  ensure  right-sizing  of  FFRDCs  and  UARCs. 

To  facilitate  consultation  with  the  Services  and  other  key  members  of  the  Department,  the 
USD(A&T)  formed  the  DoD  Advisory  Group,  chaired  by  DDR&E  with  senior  representatives 
from  OSD,  Joint  Staff,  military  departments,  and  DoD  agencies.  The  Advisory  Group  considered 
and  evaluated  alternatives  to  FFRDCs,  including  elimination  and  consolidation,  in  terms  of  risk, 
quality  and  cost.  The  Group  concluded  that  consolidation  would  not  result  in  significant  cost 
savings  and  puts  at  risk  the  focus  of  the  FFRDC  on  the  needs  of  its  sponsor.  The  Defense 
Science  Board  drew  similar  conclusions,  finding  that  unique  core  work  existed  at  each  FFRDC 
requiring  a  special  relationship  between  the  FFRDC  and  its  sponsor. 

The  DoD  implemented  the  “core”  work  concept  in  FY1995  as  a  way  to  ensure  FFRDCs  and 
UARCs  are  doing  the  kinds  of  work  they  should  be  doing.  The  core  concept  includes  defining 
core  work  (core  statement),  establishing  specific  core  criteria  to  determine  whether  a  task  is 
within  the  scope  of  the  core  statement,  and  projecting  core  workload  by  means  of  staff  years 
(STEs).  The  core  concept  has  been  refined  in  FY1996  and  was  the  basis  for  the  five-year 
projection  of  STEs  for  each  FFRDC  and  UARC.  The  driver  is  the  core  statement  which  is  an 
evolving  document  revised  periodically  to  reflect  changing  requirements  across  the  DoD. 

The  core  concept  has  become  the  foundation  of  the  revised  DoD  FFRDC  Management  Plan  and 
the  new  UARC  Management  Plan.  The  plans  implement  DoD’s  management  initiatives  by 
establishing  policies  and  procedures  that  apply  to  FFRDC  and  UARC  sponsors,  users  and 
contracting  activities  that  award  FFRDC  or  UARC  contracts.  The  plan  outlines  approval  levels 
for  specific  actions  and  establishes  consistent  reporting  requirements  across  DoD. 

The  bottom  line  is  that  DoD  believes  that  FFRDCs  and  UARCs  are  doing  high-quality,  high-value 
technical  and  analytical  work  that  could  not  be  provided  as  effectively  by  other  means.  The  DoD 
Advisory  Group  and  Defense  Science  Board  agree  and  believe  FFRDCs  (and  UARCs)  should  be 
retained  because  of  their  quality  and  the  special  relationships  they  have  with  their  sponsors  on 
matters  which  are  of  great  importance  to  the  Department.  These  FFRDCs  and  UARCs  are 
critically  important  national  assets.  They  have  provided  key  contributions  in  the  past  and  will 
address  critical  needs  now  and  in  the  future.  To  ensure  the  sustainment  of  these  contributions,  the 
Department  has  implemented  proactive  management  on  the  part  of  sponsors  and  DDR&E. 
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Appendix  B 


Center  for  Naval  Analyses 

The  written  and  oral  information  CNA  and  its  Sponsor  presented  to  the  Task  Force  is  summarized 
in  this  appendix.  The  Navy  Sponsor  presented  a  very  supportive  view  of  the  need  for  the  work 
being  done  by  the  FFRDC,  and  the  value  of  CNA  products. 

Objectives  and  guidelines  for  scope  of  work,  organizational  structure,  size,  and 
appropriateness  of  customers. 

The  Department  of  the  Navy  (DON)  established  the  Center  for  Naval  Analyses  (CNA),  its  only 
FFRDC,  to  provide  access  to  private  sector  scientific  knowledge  in  support  of  DON 
decisionmakers.  The  Center  traces  its  origin  to  1942,  making  it  the  oldest  DoD  FFRDC. 

The  CNA  has  functioned  as  an  independent  entity  with  a  contract  directly  with  the  Navy  since 
October  1,  1990.  It  restructured  in  1993  as  The  CNA  Corporation  (CNAC)  with  two  divisions: 
the  CNA  FFRDC  sponsored  by  the  Department  of  the  Navy,  and  a  new  operating  unit,  the 
Institute  for  Public  Research  (IPR)  (see  figure  B-l). 

Figure  B-l:  The  CNA  Corporation 
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Figure  B-2  shows  a  recent  funding  profile  for  the  CNA  FFRDC.  The  FY  96  DoD  ceiling 
allocation  for  the  FFRDC  is  $45.9  million.  Of  this,  $42.9  million  is  core  funding  for  the  Navy  and 
Marine  Corps  research  and  about  $3.0  million  is  for  other  DoD  research.  CNA’s  non-DoD  work 
constitutes  less  than  one  percent  of  its  FY  96  effort.  The  work  done  outside  the  FFRDC  umbrella 
and  within  the  IPR,  provides  analytical  and  support  services  to  non-Navy  clients  (these  clients  can 
also  include  DoD  non-Navy  clients). 


Figure  B-2:  CNA  Funding  Profile 
Actual  Obligations 
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SOURCE:  Department  of  Defense 
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CNA’s  DoD  ceiling  allocation  for  FY  96  represents  a  drop  of  about  19%  in  real  terms  since  FY 
91 .  The  overall  funding  limit  has  been  set  by  DDR&E.  DON  Product  Area  Managers  must 
approve  sponsor  requested  efforts  within  the  1 1  contractually  defined  areas  before  N81  and  the 
Office  of  Naval  Research  (ONR)  authorize  CNA  to  execute  projects  and  programs.  N81  and  the 
ONR  review  technical,  administrative,  and  budgetary  aspects  of  individual  projects. 

CNA’s  mission,  as  stated  in  the  latest  contract,  is  to:  provide  an  independent  authoritative  source 
of  research  and  analysis  that  is  focused  upon  the  major  present  and  future  issues  affecting  the 
Navy  and  Marine  Corps. 


CNA’s  core  competencies  are: 


Operations  Analysis:  Work  that  helps  the  fleet  or  fleet  Marine  forces  develop  or 
evaluate  new  tactics;  test  or  employ  new  equipment;  or  plan,  conduct,  or  learn  from 
exercises  or  real-world  operations. 
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System  Requirements  and  Acquisition:  Work  that  addresses  the  potential  utility  of 
new  technologies;  the  relative  need  for  new  systems  or  capabilities;  or  the  costs  and 
consequences  of  acquiring  a  particular  system  or  family  of  systems. 

Resource  Analysis:  Studies  or  analyses  of  issues  involving  manpower  and  training, 
readiness  and  logistics,  or  installations  and  infrastructure. 

Program  Planning:  Analyses  intended  to  help  DON  formulate  coherent  and 
executable  long-term  plans  covering  the  evolution  of  naval  missions;  the  integration 
or  improvement  of  naval  capabilities;  the  acquisition  of  needed  supplies  or  services; 
or  the  future  size  and  shape  of  naval  forces. 

Policy,  Strategy,  and  Doctrine:  Broad  studies  intended  to  help  DON  develop, 
promulgate,  evaluate,  or  refine  new  policies,  strategies,  or  doctrine. 

CNA’s  core  work  is  defined  as  work  that  appropriately  integrates  CNA’s  mission,  purpose,  and 
capabilities;  its  core  competencies;  and  the  characteristics  of  a  strategic  FFRDC  relationship  with 
its  sponsor.  The  test  of  appropriateness  requires  that  the  work  being  assigned:  (1)  is  consistent 
with  CNA’s  mission,  purpose,  and  capabilities,  (2)  requires  the  characteristics  of  a  strategic 
relationship,  (3)  requires  at  least  one  of  CNA’s  core  competencies,  (4)  could  not  be  done  as 
effectively  elsewhere,  (5)  is  free  from  real  or  perceived  conflicts  of  interest,  (6)  could  not  be  done 
by  available  government  resources,  (7)  will  be  done  mainly  by  CNA  employees,  and  (8)  is  not  an 
inherently  governmental  function. 

CNA  has  a  research  staff  of  230.  CNA  reports  that  its  scientific  staff  possesses  a  level  of 
education  and  experience  across  a  broad  range  of  fields  and  disciplines.  More  than  95  percent  of 
the  CNA  research  staff  have  advanced  degrees,  about  66  percent  with  PhDs.  Additionally, 

CNA’s  research  staff  averages  16  years’  experience  since  receiving  their  bachelor’s  degree  and 
averages  12  years  of  experience  related  to  CNA  studies.  CNA  attempts  to  maintain  an  essential 
critical  mass  of  authoritative  knowledge  and  skills  in  various  research  areas.  The  strategic  nature 
of  CNA’s  relationship  with  DON  has  encouraged  the  retention  of  experienced  analysts  who 
specialize  in  naval  matters  and  related  technical  subjects. 

CNA  personnel  work  with  DON  personnel  both  in  Washington  and  at  30  Navy,  6  Marine,  and  5 
Joint  Commands.  The  FFRDC  and  the  sponsor  both  stress  the  importance  of  the  close  and  long¬ 
term  relationship  between  the  FFRDC  and  DON  personnel. 

CNA  reports  that  the  CNA  Field  Representatives  (about  40  of  whom  work  at  24  locations  around 
the  world)  provide  on-scene  technical  analytic  support  that  is  focused  on  the  command’s  priority 
issues  and  provide  support  for  contingency  planning  in  crisis  situations.  The  field  analysts  provide 
analytical  support  during  real  world  operations;  help  design,  reconstruct,  and  analyze  more  than  a 
dozen  major  exercises  per  year;  and  assist  in  evaluating  new  equipment,  developing  and  testing 
new  tactics,  and  refining  or  revising  naval  doctrine. 
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CNA  sees  this  field  work  as  an  added  benefit  for  DON  since  field  representatives’  exposure  to 
fleet  operations  adds  to  the  realism  and  relevance  of  the  studies  they  do  when  they  return  to  CNA 
headquarters  from  their  2-year  field  tours. 

The  CNA  FFRDC  will  perform  only  core  work  for  DON  and  other  DoD  activities.  Non-core 
work  will  not  be  conducted  at  CNA  but  can  be  done  at  IPR.  CNA  and  its  Navy  sponsor  have 
developed  criteria  (outlined  above  in  the  discussion  of  core  work)  to  determine  if  work  is 
appropriate  and  core  for  CNA. 

The  level  and  appropriateness  of  non-DoD  work  and  work  for  other  than  the  primary  sponsor. 

CNA  reports  that  the  only  non-DoD  work  currently  being  conducted  in  the  FFRDC  is  a  study  for 
the  U.S.  Coast  Guard.  Non-DoD  work  is  very  limited  and  is  closely  monitored  by  the  sponsor. 

Non-core  work  may  be  performed  by  the  Institute  for  Public  Research  (DPR),  the  non-FFRDC 
division  of  The  CNA  Corporation.  The  following  are  the  criteria  for  acceptance  of  work  by  IPR: 
(1)  does  not  pose  an  actual  or  potential  conflict  of  interest  with  FFRDC  work,  (2)  will  not 
interfere  with  performance  of  FFRDC  work,  (3)  may  draw  on  Corporation’s  technical  expertise 
but  not  specific  subject  matter  of  FFRDC  work,  (4)  must  be  done  in  accordance  with  applicable 
cost  accounting  standards  and  accounting  principles  and  will  not  cause  non-compliance  in  the 
FFRDC,  (5)  work  for  a  DoD  component  may  not  be  within  FFRDC ’s  purpose,  mission,  general 
scope  of  effort  or  scope  of  core  work,  and  (6)  must  be  fully  funded  by  sponsor  or  fees  earned  by 
the  Corporation. 

Compliance  with  the  DoD  Management  Plan 

The  DoD  Management  Plan  directs  that  the  S&A  FFRDCs  (1)  maintain  a  relatively  stable  level- 
of-effort,  and  (2)  maintain  competency  in  core  areas.  Staff  years  of  technical  effort  (STEs)  are 
used  in  sizing  and  managing  DoD-funded  FFRDC  work. 

The  Plan  states  that  DoD  FFRDCs  may  only  perform  core  work,  that  such  work  must  be 
approved  by  the  primary  sponsor;  can  only  be  accomplished  for  DoD,  other  government  entities 
(including  state  and  municipal  governments)  and  for  not-for-profit  activities;  and  cannot  include 
commercial  work. 

Parent  institution  operating  DoD  FFRDCs  may  perform  non-FFRDC  work,  but  such  work  must 
conform  to  some  specific  guidelines:  be  in  the  national  interest;  not  undermine  the  independence, 
objectivity,  and  credibility  of  the  institute  or  call  into  question  the  freedom  from  conflict  of 
interest;  not  take  unfair  advantage  of  FFRDC  work;  and  normally  work  in  the  public  sector, 
although  commercial  work  may  be  accepted  if  the  sponsor  agrees. 

Fees  may  be  authorized.  The  Plan  calls  for  “an  annual  fee  proposal  justifying  each  element  of 
fee.” 
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CNA  maintained  a  relatively  stable  level-of-effort  until  FY  95  when  it  experienced  a  sharp 
reduction  in  its  DoD  funding  allocation.  The  Center  reports  that  it  maintains  a  critical  mass  in 
areas  of  particular  interest  to  its  sponsor  (although  the  1995  5 -year  review  indicates  that  the 
medical  area  appears  to  be  a  problem).  As  noted  above,  CNA  and  its  sponsor  have  defined  five 
areas  of  core  competencies.  These  are  detailed  in  the  current  contract.  CNA  management, 
however,  expresses  concern  about  their  future  ability  to  maintain  a  critical  mass  in  key  areas 
unless  CNA’s  DoD  funding  ceiling  is  increased  above  the  FY  95-96  level. 

The  FFRDC  reports  that  it  has  developed  a  deep  understanding  of  Navy  and  Marine  Corps 
problems  through  its  54  years  of  Service  association  and  such  programs  as  the  on-site  analysts.  It 
uses  this  understanding  to  identify  and  address  Navy  and  Marine  Corps  work  that  could  not  be 
accomplished  by  other  organizations. 

CNA  reports  it  does  only  core  work  in  the  FFRDC.  The  work  done  in  the  IPR  is  screened 
according  to  the  criteria  already  outlined.  CNA  provides  the  sponsor  quarterly  reports  of  all 
contracts,  grants,  etc.,  of  the  IPR. 

Sponsor’s  Management  Processes 

The  relationship  between  the  DON  and  CNA  is  governed  by  the  Office  of  Federal  Procurement 
Policy  directives  and  the  Federal  Acquisition  Regulation,  instructions  of  the  Secretary  of  Defense 
concerning  FFRDCs,  the  sponsoring  agreement  between  the  DON  and  CNA  and,  of  course, 
pertinent  legislation.  The  CNA  oversight  chain  is  shown  in  figure  B-3. 

The  DON’s  Policy  Council  is  headed  by  the  Assistant  Secretary  of  the  Navy  (Research, 
Development  and  Acquisition)  and  includes  the  other  Assistant  Secretaries  of  the  Navy,  the  Navy 
General  Counsel,  and  pertinent  Deputy  Chiefs  of  Naval  Operations  and  comparable  Marine  Corps 
general  officers.  The  Policy  Council  reviews  and  approves  CNA’s  work  annually,  and  approves 
the  level  of  resources  and  the  plan  of  work  for  the  coming  year. 

The  Navy  and  Marine  Corps  Scientific  Officers  are  responsible  for  the  overall  coordination  of 
CNA’s  research  program  within  those  Services.  The  Navy’s  Deputy  Scientific  Officer,  is  also  the 
Contracting  Officer’s  Technical  Representative  (COTR),  and  is,  therefore,  responsible  in  detail  for 
CNA’s  research  work.  The  DON’s  Contracting  Officer  in  the  Office  of  Naval  Research,  is 
responsible  for  the  overall  administration  of  CNA’s  contract. 

CNA’s  funding  and  research  programs  are  divided  into  seven  product  areas  and  four  levels  of 
effort  areas.  Each  product  area  has  a  three-star  Navy  or  Marine  officer,  or  a  civilian  Assistant 
Secretary,  called  a  “DON  Product  Area  Manager,”  who  interacts  with  a  CNA  Vice  President  in 
discussing  and  deciding  upon  the  annual  research  program  for  that  product  area.  The  Product 
Area  Managers  shown  in  the  figure  are  allocated  a  dollar  ceiling  for  the  product  area  and 
prioritize  the  work  within  that  area.  They  are  responsible  for  review,  approval,  and  coordination 
of  all  the  work  in  their  area.  Each  Product  Area  Manager  approves,  in  writing,  each  separate 
project  assigned  to  CNA  during  the  year.  In  addition,  the  Marine  Corps  for  over  five  years  has 
convened  a  quarterly  review  conference  between  Marine  staff  and  CNA  at  the  flag  officer  level  to 
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discuss  progress  and  issues.  The  Navy  has  recently  decided  to  convene  a  similar  conference  each 
quarter.  There  are  also  more  frequent  working  meetings  between  CNA  analysts  and  the  DON 
sponsors  of  each  project. 

Figure  B-3:  Department  of  the  Navy  Management  Oversight 


Policy  Cotncil 


-  Assistant  Secretary  of  the  Navy  (RD&A) 

-  Assistant  Secretary  of  the  Navy  (M&RA) 

-  Assistant  Secretary  of  the  Navy 

(installations  &  Environment) 

-  Assistant  Secretary  of  the  Navy  (Financial 

Management 

*  General  Counsel  of  the  Navy 

-  Deputy  Chief  of  Naval  Operations  (R,  WR&A) 

-  Director  of  Test  &  Evaluation  &  Technology 

Requirements 

♦  Chief  of  Naval  Research 

Director,  Office  of  Program  Appraisal 


Commanding  General,  Marine  Corps  Combat 
Development  Command 
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of  the  Navy 
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SOURCE:  Department  of  the  Navy 
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CNA  reports  in  writing  each  quarter  on  the  cost  and  performance  of  each  project  under  way. 

CNA  reports  it  is  scrutinized  rigorously  and  regularly  by  various  audit  and  investigating  agencies 
as  well — the  DoD  Inspector  General,  the  General  Accounting  Office,  the  Naval  Audit  Service, 
and  the  Defense  Contract  Audit  Agency  are  the  more  prominent.  CNA  reports  it  had  16  audits  by 
these  agencies  in  FY  1994.  CNA’s  security  arrangements  are  audited  by  DoD  regularly  as  well. 
The  Department  of  Labor  also  audits  The  CNA  Corporation  from  time  to  time.  (For  its  part,  the 
CNA  Corporation  employs  a  private  accounting  firm,  Grant-Thomton,  as  its  own  external 
auditor,  reporting  to  the  Board  of  Trustees.) 

In  considering  award  fees  an  assessment  of  CNA’s  effectiveness  and  efficiency  is  made  as  a  part 
of  the  evaluation  of  the  contractor’s  performance.  Factors  that  are  considered  in  the  overall 
evaluation  are:  Timeliness,  Quality,  and  Cost  Effectiveness  and  Efficiency.  Timeliness  constitutes 
25%  of  the  score.  Quality  is  55%,  and  Cost  Effectiveness  and  Efficiency  is  20%. 

Metrics  that  are  used  in  this  evaluation  are  shown  in  tables  B-l,  B-2,  and  B-3 . 

Table  B-l:  Evaluation  Metrics 


Evaluation  Factor 

Evaluation  Elements 

Weighting  Factor 

Timeliness 

Submission  of  Project  Proposals  and  Revisions 

5% 

Adherence  to  Project  Milestones  and  Final  Delivery  Requirements 

10% 

Responsiveness  to  Sponsor  Needs 

10% 

Quality 

Demonstrated  Understanding  of  Sponsor’s  Technical  Needs  and 
Mission  Requirements 

15% 

Accuracy,  Thoroughness,  Innovativeness,  and  Objective  of 

Research 

20% 

Cunency  of  Expertise  and  Knowledge/Use  of  Appropriate  Research 
Methodologies 

20% 

Cost  Effectiveness  and 
Efficiency 

Control  of  Indirect  Costs 

10% 

Manages  to  Budgets 

10% 
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Table  B-2:  Grading  Table  -  Timeliness  and  Quality  Evaluation  Factors 


Adjective  Grade 

Description 

Point  Range 

Superior 

Superlative  Level  of  Performance,  Achievement  of  Distinguished  Results 
and  Effectiveness.  No  Deficiencies 

91-100 

Excellent 

Of  Exceptional  Merit;  Exemplary  Performance  in  a  Timely,  Efficient  and 
Economical  Manner  -  Very  Minor  Deficiencies  -  No  Effect  on  Overall 
Performance 

81-90 

Good 

Very  Effective  Performance;  Fully  Responsive  to  Contract  Requirements; 

More  than  Adequate  Results;  Reportable  Deficiencies,  but  with  Little 
Identifiable  Effect  on  Overall  Performance 

71-80 

Satisfactory 

Effective  Performance;  Responsive  to  Contract  Requirements;  Adequate 
Results.  Reportable  Deficiencies  with  Identifiable,  but  Not  Substantial, 

Effects  on  Overall  Performance 

61-70 

Marginal 

Meets  or  Slightly  Exceeds  Minimum  Acceptable  Standards;  Useful  Levels  of 
Performance,  but  Suggest  Remedial  Action,  Reportable  Deficiencies  which 
Adversely  Affect  Overall  Performance. 

51-60 

Unsatisfactory 

Below  Minimum  Acceptable  Standards;  Poor  Performance;  Inadequate 
Results;  Requires  Prompt  Remedial  Action.  Significant  Deficiencies. 

0 

Table  B-3:  Grading  Table  -  Cost  Effectiveness  and  Efficiency  Evaluation  Factor 


Adjective  Grade 

Description 

Point  Range 

Superior 

Superlative  Level  of  Performance;  Achievement  of  Distinguished  Results 
and  Effectiveness.  Actual  Rates  for  Indirect  Costs  are  Less  than  Budget. 

All  Projects  are  Performing  within  Budget 

91-100 

Excellent 

Of  Exceptional  Merit;  Exemplary  Performance.  Actual  Indirect  Rates  are 
within  Budget.  There  are  only  Minor  Variances,  less  than  10%,  in  a  Minority 
of  Project  Budgets,  and  No  Project  Varies  by  More  than  10% 

81-90 

Good 

Very  Effective  Performance;  More  than  Adequate  Results.  Actual  Indirect 
Cost  Rates  are  within  1  %  of  Budget.  Variance  in  Project  Budgets  by  Less 
Than  1 0%  on  a  Majority  of  Projects,  and  No  Project  Varies  by  More  Than 
10%. 

71-80 

Satisfactory 

Effective  Performance;  Adequate  Results.  Actual  Indirect  Cost  Rates  are 
within  5%  of  Budget.  Variance  in  Project  Budgets  by  More  Than  10%  on  a 
Few  Projects,  and  No  Project  Varies  by  More  Than  15%. 

61-70 

Marginal 

Meets  or  Slightly  Exceeds  Minimum  Acceptable  Standards.  Actual  Indirect 
cost  Rates  are  within  10%  of  Budget.  Variance  in  Project  Budgets  by  More 
Than  10%  on  a  Significant  Number  of  Projects  and  at  Least  One  Project 
Varies  by  More  Than  15%. 

51-60 

Unsatisfactory 

Below  Minimum  Acceptable  Standards;  Poor  Performance;  Inadequate 
Results;  Requires  Prompt  Remedial  Action.  Actual  Indirect  Costs  Exceed 

1 0%  of  Budget  in  One  or  more  Rates.  Variance  in  Project  Budgets  by  More 
Than  10%  on  a  Majority  of  Projects,  and  two  or  more  Projects  Vary  by  More 
Than  15%. 

0 

The  Thoroughness  of  the  Five-Year  Review 

The  Comprehensive  Review  for  the  Center  for  Naval  Analyses  conducted  by  the  sponsor 
is  lengthy.  The  criteria  established  by  FAR  35.017  were  used  to  determine  the  continued 
usefulness  of  CNA  and  its  effectiveness.  These  are:  (1)  examination  of  the  DON’S 
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technical  needs  and  mission  requirements  that  are  served  by  CNA  as  an  FFRDC,  (2) 
consideration  of  alternative  sources  to  meet  the  DON’S  needs,  (3)  an  assessment  of 
CNA’s  effectiveness  and  efficiency  in  meeting  the  DON’S  needs,  (4)  an  assessment  of  the 
adequacy  of  CNA’s  management  in  ensuring  a  cost  effective  operation;  and  (5) 
determination  that  the  criteria  for  establishing  CNA  continue  to  be  satisfied  and  that  the 
sponsoring  agreement  is  in  compliance  with  the  FAR  35.017-1. 

The  August  1995  Review  details  DON’S  technical  needs  and  mission  requirements  that  are 
currently  served  by  CNA.  The  needs  and  mission  requirements  are  presented  in  broad  terms. 

The  discussion  of  whether  alternative  sources  can  meet  DON’S  needs  is  very  general,  very  short, 
and  highly  aggregated.  The  Review  considers  four  categories:  in-house  and  other  government 
resources;  universities;  commercial  industry;  other  nonprofits,  and  other  FFRDCs.  Each  of  these 
categories  is  shown  to  have  some  of  the  characteristics  of  CNA  but  is  found  to  be  deficient  in  at 
least  one  area  and  is  presented  as  not  competitive  with  CNA.  For  example: 

•  Government  sources  lack  “a  depth  and  breadth  of  knowledge  which  stems 
from  a  long  term  relationship.” 

•  University  sources  “would  not  possess  the  necessary  experience  to  effectively 
perform  the  work  assigned  to  CNA.” 

•  Commercial  Industry  “without  a  guarantee  of  continuous  funding  and 
involvement,  commercial  entities  cannot  afford  to  dedicate  the  resources 
necessary  to  maintain  currency  in  Naval  operations,  policies,  doctrine  and 
capabilities.” 

•  As  for  other  Nonprofits  and  FFRDCs,  “while  the  Navy  maintains  a  relationship 
with  other  systems  engineering  and  research  laboratory  FFRDCs  and  many 
nonprofits,  none  of  these  relationships  provide  the  detailed  knowledge  and 
expertise  in  Naval  operations,  policies,  doctrine,  and  capabilities  to  perform  the 
work  assigned  to  CNA.” 

The  effectiveness  and  efficiency  evaluation  of  CNA  that  appears  in  the  5-year  Review  is  the  result 
of  a  questionnaire  sent  to  Navy  Product  Area  Managers.  The  Review  includes  a  number  of 
statements  on  savings  that  have  occurred  as  a  result  of  CNA  studies.  The  Review  notes  that  the 
Navy  comments  on  CNA  work  with  some  exceptions  (e.g.,  medical  and  training)  were 
“extremely  positive.” 

The  assessment  of  CNA’s  management  in  ensuring  a  cost  effective  operation  that  appears  in  the 
5-year  Review  consists  largely  of  discussions  of  compliance  with  auditing  rules.  There  is  a  limited 
comparison  of  overall  costs  of  man-years  with  those  of  other  FFRDCs  (figure  B-4). 
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Figure  B-4  :  Average  Cost  per  MTS  Year  for  DoD  Sponsored 
Study  and  Analysis  FFRDCs  in  FY  94 
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The  Review  also  shows  a  reduction  in  the  average  cost  of  CNA  MTS  hours  (figure  B-5).  There  is 
no  comparison  with  for-profits  organizations  or  with  non-FFRDC,  non-profit  organizations. 

Figure  B-5:  CNA  Average  Cost  per  MTS  Hour 
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Appendix  C 


The  Institute  for  Defense  Analyses 

The  written  and  oral  information  of  the  Institute  of  Defense  Analyses  (IDA)  and  its  Sponsors 
presented  to  the  Task  Force  is  summarized  in  this  appendix.  The  primary  sponsors  of  the  two 
DoD  FFRDCs  and  the  primary  user  of  the  operational  evaluation  division  were  very  supportive  of 
the  need  for  the  work  being  done  by  the  FFRDCs  and  the  value  of  IDA  products. 

Objectives  and  guidelines  for  scope  of  work,  organizational  structure,  size,  and 
appropriateness  of  customers 

IDA  was  formed  in  1956  at  the  request  of  the  Secretary  of  Defense  to: 

“...assist  the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the  Unified 
Commands  and  Defense  Agencies  in  addressing  important  national  security 
issues,  particularly  those  requiring  scientific  and  technical  expertise.” 

The  Institute  was  originally  chartered  by  a  consortium  of  universities  led  by  MIT .  It  became  an 
independent  corporation  in  1 968.  Over  the  years,  EDA  has  modified  its  structure  in  response  to 
sponsor  needs  and  requests  and  currently  manages  two  FFRDCs,  the  Studies  and  Analyses  Center 
and  the  Communications  and  Computing  FFRDC.  The  current  organizational  structure  is  shown 
in  figure  C-l. 

The  Communications  and  Computing  Center  is  sponsored  by  the  National  Security  Agency.  It 
includes  the  Center  for  Communications  Research  at  Princeton  established  in  1958;  the  Center  for 
Computing  Sciences  at  Bowie  established  in  1 984;  and  the  Center  for  Communications  Research 
at  La  Jolla  established  in  1989. 

EDA’s  Studies  and  Analyses  FFRDC  is  sponsored  by  the  Under  Secretary  of  Defense  (A&T),  with 
the  Director,  Acquisition  Program  Integration  having  day-to-day  management  and  oversight 
responsibility.  The  Studies  and  Analyses  Center  has  six  divisions  supporting  OSD  as  a  whole, 
including  DARPA. 

The  Operational  Evaluation  Division  is  managed  under  the  Studies  and  Analyses  FFRDC,  and 
works  exclusively  for  the  OSD,  Director,  Operational  Test  and  Evaluation  (DOTE).  The  Division 
has  been  providing  about  70  MTS/year  for  DOTE. 
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Figure  C-l:  Institute  for  Defense  Analyses 


SOURCE:  Institute  for  Defense  Analyses 

Each  of  IDA’s  two  FFRDCs  has  its  own  sponsor  and  contract  arrangement.  Funding  for  the 
Communications  and  Computing  Center  is  administered  through  the  National  Security  Agency. 
Funding  for  the  Studies  and  Analyses  FFRDC  comes  from  a  variety  of  sources  within  OSD,  the 
Joint  Staff,  Defense  Agencies,  and  some  limited  non-DoD  government  work.  According  to  the 
latest  available  Comprehensive  Review  (August  10,  1993),  the  majority  of  non-DoD  funding 
came  from  the  FBI  and  NASA. 

IDA’s  recent  DoD  Funding  Profile  is  shown  in  figure  C-2 


Figure  C-2:  IDA  Funding  Profile  Actual  Obligations 


120 


FY 


SOURCE:  The  Department  of  Defense 
IDA’s  DoD  FFRDCs  are  discussed  below. 

IDA  Studies  and  Analyses 

Objectives  and  guidelines  for  scope  of  work,  organizational  structure,  size,  and 
appropriateness  of  customers. 

The  IDA  Studies  and  Analyses  Center  is  the  direct  descendent  of  several  of  the  groups  originally 
created  to  support  the  DoD’ s  Weapons  System  Evaluation  Group  and  later  ARPA.  It  is  one  of 
several  studies  and  analyses  FFRDCs  supporting  the  OSD.  It  also  works  for  the  Joint  Staff,  the 
unified  commands,  and  defense  agencies.  The  mission  of  the  S&A  Center  remains  similar  to  the 
earlier  mission: 

. .  .to  assist  the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the  unified  commands, 
and  defense  agencies  in  addressing  important  national  security  issues,  particularly  those 
requiring  scientific  and  technical  expertise.  IDA  research  requires  an  in-depth  knowledge 
of  Defense  systems,  technologies,  operations,  strategies,  and  resources. 

The  FFRDC’s  organizational  structure  was  shown  in  figure  C-l  above.  Its  stated  core 
competencies  are: 
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Systems  Evaluations.  Evaluations  support  decisions  on  acquisition  and  program  planning,  and 
involve  assessments  of  military  worth,  system  performance,  technological  risks,  costs,  and  allied 
interoperability. 

Technology'  Assessments.  Provides  scientific,  technical,  and  analytical  support  related  to 
identifying,  developing  and  using  advanced  technologies.  This  involves  assessments  of 
technology  feasibility,  performance,  productibility,  demonstrations,  and  development  risks. 

Force  and  Strategy  Assessments.  IDA  conducts  assessments  relating  systems,  operational 
performance,  force  structure,  and  national  security  strategy. 

Resource  and  Support  Analyses.  IDA  develops  methods  and  models  for  estimating  the  costs  to 
develop,  procure,  test,  operate,  and  support  Defense  forces  and  systems.  IDA  applies  these 
techniques  when  evaluating  the  resource  consequences  of  Defense  policy,  planning,  programming 
and  acquisition  decisions. 

The  relation  of  these  core  competencies  to  on-going  research  activities  are  shown  in  table  C-l. 


Table  C-l:  IDA  Core  Competencies 


Systems  Evaluations 

•  Strategic  Systems  &  Missile  Defense 

•  Tactical  Systems 

•  C3,  Surveillance  and  Space  Systems 

•  Information  and  Computing  Systems 

•  Test  &  Evaluation 


Force  and  Strategy  Assessments 

•  Joint  and  Combined  Force  Planning, 
Operations  &  Assessments 

•  National  Security  Strategy  Issues 


Technology  Assessments 

•  Materials 

•  Space,  Air,  and  Missile  Technologies 

•  Sensors,  Surveillance,  and  Target  Acquisition 

•  Simulation 

•  Information  and  Computing  Technologies 

•  Manufacturing 

•  Technology  Planning  and  Strategy 

•  International  Technology-  Planning  &  Controls 

Resource  and  Support  Analyses 

•  Cost  Analyses 

•  Industrial  Base  and  Mobilization 

•  Acquisition  Planning  &  Resource  Management 

•  Training  and  Readiness 

•  Environmental  Technologies _ 


SOURCE:  Institute  for  Defense  Analyses 

Core  work  is  defined  as  that  which  is  appropriate  for  IDA  to  conduct  in  pursuit  of  IDA’s  mission 
and  charter  to  support  DoD,  and  in  light  of  the  strategic  relationship  maintained  between  IDA  and 
its  sponsors.  This  research  must:  (1)  be  consistent  with  EDA’s  mission,  purpose,  and  capabilities; 
(2)  be  consistent  with  DoD’s  needs  for  IDA  work  as  reflected  by  the  core  competencies  that  IDA 
maintains,  (3)  require  the  long-term  FFRDC  special  relationship  between  sponsors  and  EDA,  and 
(4)  meet  an  important  DoD  need. 
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IDA  reports  that  its  customers  are  sensitive  to  core  competencies  and  there  is  little  or  no  pressure 
for  non-core  work.  IDA  reports  that  in  the  past,  some  core  work  has  developed  to  the  point  that 
the  special  FFRDC  relationship  and/or  the  core  competencies  of  IDA  are  no  longer  required. 
Examples  of  work  terminated  or  reduced  include: 

•  Support  of  developed  information  systems 

•  Expert  systems  for  counter-terrorism,  and  counter-drug  data 

•  Support  for  Armed  Forces  Information  Service 

•  Operating  and  maintaining  particular  models 

In  1995,  DoD  sponsored  about  377  MTS  within  the  fiscal  ceiling  applied  to  IDA’s  S&A  FFRDC. 
The  composition  of  academic  disciplines  of  the  research  staff  of  the  S&A  FFRDC  is  shown  in 
figure  C-3 . 


Figure  C-3:  Composition  of  the  IDA  S&A  Research  StafT 
Phds  57%,  Masters  36% 


SOURCE:  Institute  for  Defense  Analyses 

According  to  the  sponsor  briefing,  the  FY  1996  ceilings  are  286  MTS  for  S&A  and  67  MTS  for 
T&E  (subsequently  changed  to  a  total  of  352.5  STE). 

Fifty-seven  percent  of  the  research  staffhold  doctorates  and  another  36  percent  have  master  s 
degrees. 
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The  S&A  FFRDC’ s  funding  comes  from  sponsor  studies  funds  and  program  funds.  There  is  no 
specific  IDA  program  budget  line.  Study  funds  are  justified  by  individual  sponsors  and  they 
decide  what  work  needs  to  be  done  and  who  will  do  it.  A  breakout  of  the  funding  sources  for  FY 
1995  is  shown  at  figure  C-4. 

Figure  C-4:  S&A  FFRDC  1995  Funding  Sources 


Commission  on 
Roles  &  Missions  6% 
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SOURCE:  Institute  for  Defense  Analyses 

Funding  for  the  S&A  FFRDC  is  shown  in  table  C-2. 

Table  C-2  :  S&A  Funding 
($  millions) 

FY  1991  FY  1992  FY  1993  FY  1994  FY  1995 

IDA  Studies  and  Analyses  67.3  63.7  71.9  72.0  68.4 

Excludes  Commission  on  Roles  and  Missions  funds  outside  of  ceiling  in  FY  1995  as  directed  by  Congress. 

SOURCE:  Institute  for  Defense  Analyses 

IDA  performs  no  non-core  work.  The  stated  work  assignment  criteria  conforms  to  that  of  all  the 
FFRDCs.  The  work  assignment  principles  briefed  by  the  primary  sponsors  are  shown  in  table  C- 
3. 
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Table  C-3:  Work  Assignment  Principles 

•  Consistent  With  IDA  Mission  and  Capabilities 

•  Consistent  With  IDA  Core  Competencies 

-  Systems  evaluations 

-  Technology  assessments 

-  Force  and  strategy  assessments 

-  Resource  and  support  analyses 

•  Require  a  Strategic  Relationship 

-  Long-term  continuity 

-  Conflict-free  environment 

-  Responsiveness 

•  Meet  an  Important  DoD  Need 

Each  sponsor  maintains  its  own  task  formulation  and  approval  process  —  all  embody  multiple 
reviews  by  senior  officials _ _ _ _ _ 

SOURCE:  Office  of  the  Secretaiy  of  Defense 

The  primary  sponsor  noted  that  each  of  the  principal  work  sponsors  has  its  own  process  for 
assigning  work  to  IDA  and  provided  some  examples.  The  President  of  IDA  generalized  the 
process  in  his  briefing  (see  box  C-l).  He  noted  that  the  process  has  become  increasingly  complex 
over  the  past  five  years.  He  said  that  it  is  more  complex  and  time  consuming  than  many 
arrangements  with  for-profits. 

Box  C-l:  Typical  Work  Assignment  Process 

•  Sponsor  provides  a  work  statement  and  FFRDC  justification 

•  IDA  works  with  sponsor  to  prepare  task  order 

•  IDA  VPs  and  Research  Division  Director  review  task  order  for  appropriateness 

•  SES-level  sponsor  approves  the  task  order  and  FFRDC  justifications 

•  Contracting  officer’s  representative  (OSD  FFRDC  Programs  Office)  reviews  task  order 
for  appropriateness  and  consistency  with  IDA  core  competencies 

•  Contracting  office  reviews  task  order  and  funding  documents 

•  Competition  advocate  reviews  justifications  for  use  of  FFRDC 

•  Director  FFRDC  Programs  and  President  of  IDA  sign  task  order _ 


-  Familiarity  with  sponsors 

-  Objective,  high-quality  work 

-  Proprietary/sensitive  data 
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The  level  and  appropriateness  of  non-DoD  work  and  work  for  other  than  the  primary  sponsor 

IDA  reports  it  seeks  no  non-core  work  or  non-FFRDC  work.  It  does  a  modest  amount  of  non- 
DoD  work.  Any  non-sponsor  work  must  be  approved  in  writing  by  IDA’s  sponsor.  The  criteria 
for  accepting  such  work  is  that  it: 

•  Is  national  security  work 

•  Develops  and/or  improves  the  capability  to  support  DoD  sponsor 

•  Represents  no  diversion  of  key  talents  from  DoD  sponsor  work 

IDA  presented  three  examples: 

•  NASA  space  support  system 

•  FBI  counter-terrorism,  counter  drug 

•  DOE  defense  programs 

Compliance  with  the  DoD  Management  Plan 

The  DoD  Management  Plan  directs  that  the  S&A  FFRDCs  (1)  maintain  a  relatively  stable  level- 
of-effort,  and  (2)  maintain  competency  in  core  areas.  Staff  years  of  technical  effort  (STEs)  are 
used  in  sizing  and  managing  DoD-funded  FFRDC  work. 

The  Plan  states  that  DoD  FFRDCs  may  only  perform  core  work,  that  such  work:  ‘must  be 
approved  by  the  primary  sponsor;  can  only  be  accomplished  for  DoD,  other  government  entities 
(including  state  and  municipal  governments)  and  for  not-for-profit  activities;  cannot  include 
commercial  work. 

Parent  institutions  operating  DoD  FFRDCs  may  perform  non-FFRDC  work,  but  such  work  must 
conform  to  some  specific  guidelines:  be  in  the  national  interest;  not  undermine  the  independence, 
objectivity  and  credibility  of  the  institute  or  call  into  question  the  freedom  from  conflict  of 
interest;  and  not  take  unfair  advantage  of  FFRDC  work.  The  work  must  normally  be  in  the  public 
sector,  although  commercial  work  may  be  accepted  if  the  sponsor  agrees. 

Fees  may  be  authorized.  The  Plan  calls  for  “an  annual  fee  proposal  justifying  each  element  of 
fee.” 

IDA  reports  it  continues  to  respond  to  changes  in  user  needs  for  the  Studies  and  Analyses 
FFRDC.  It  adjusts  to  ensure  “the  right  mix  of  high  quality  researchers  in  core  competency  areas.” 
Research  staff  is  projected  to  fall  between  FY  95  and  FY  96,  but  the  DoD  currently  projects  the 
STE  for  IDA’s  S&A  FFRDC  to  increase  slightly  between  FY  96  and  FY  2000. 

The  FFRDC  conducts  only  core  work.  Non-sponsor  work  is  approved  by  the  primary  sponsor. 

IDA  does  not  perform  any  non-FFRDC  work. 
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Fees  are  authorized.  The  past  use  of  fees  is  shown  in  figure  C-5. 


Figure  C-5:  Use  of  Fees  1992-1995 


SOURCE:  Institute  for  Defense  Analyses 
Sponsors  ’  Management  Process 

The  primary  sponsor  for  EDA’s  Studies  and  Analyses  FFRDC  is  the  Under  Secretary  of  Defense 
(A&T).  The  Director,  Acquisition  Program  Integration,  has  day-to-day  management  oversight 
responsibility.  The  Director  of  the  Office  of  Test  and  Evaluation,  OSD  has  technical  oversight 
over  the  test  and  evaluation  work  conducted  in  IDA’s  Operational  Evaluation  Division. 

The  Director,  Acquisition  Program  Integration  provides  management  through  the  FFRDC 
Program  Office  and  the  Contracting  Officer’s  Technical  Representative.  The  IDA  Advisory 
Board,  chaired  by  the  Director,  Acquisition  Program  Integration  and  including  senior  officials 
from  other  principal  OSD,  Joint  Staff,  and  Defense  Agency  sponsors,  recommends  the  multi-year 
level  of  professional  effort,  provides  information  on  DoD  research  needs,  reviews  research, 
evaluates  IDA  management,  and  helps  IDA  resolve  problems.  A  graphical  description  of  the 
Research  Plan  and  Program  Process  is  shown  in  figure  C-6. 
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Figure  C-6:  Research  Plan  and  Program  Process 
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A  graphical  description  of  Research  Program  Quality  Control  is  shown  in  figure  07. 

Figure  C-7:  Research  Program  Quality  Control 
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EDA  also  reports  attention  to  costs.  The  metrics  used  to  evaluate  success  in  achieving  cost 
control  include:  support  to  research  ratio  reduced  and  burden  rate  reduced.  EDA  reports  the 
“Bottom  line  result:  Man-Year  cost  in  constant  dollars  down  significantly  over  past  five  years.” 
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The  Thoroughness  of  the  Five-Year  Reviews 

IDA’s  last  Five-Year  Review  was  completed  on  August  10,  1993.  The  review  was  based  on  two 
surveys  of  the  sponsor  of  work  performed  by  IDA  and  a  review  of  the  Defense  Contract  Audit 
Agency  (DCAA)  audit  reports  for  the  last  five  years. 

The  Comprehensive  Review  reported  that  the  changing  nature  of  global  security  in  a  post-cold 
war  period  has  increased  the  requirement  “for  thorough,  in-depth  analyses  that  are  unbiased  and 
credible.” 

The  Review  listed  three  broad  alternative  sources:  in-house,  other  FFRDCs  (specifically  RAND), 
and  for-profit  contractors.  The  Review  stated  that  such  “resources  are  complementary  and  are 
used  when  appropriate;  however,  they  are  not  a  substitute  for  IDA’s  broad  based  cadre  of 
recognized  technical  and  analytical  experts.”  Subsequent  paragraphs  dismissed  “in-house” 
capabilities  as  “not  available;”  the  use  of  other  FFRDCs  were  viewed  as  impractical  because  the 
efforts  would  have  become  fragmented  because  “there  is  no  other  single  FFRDC  or  group  of 
FFRDCs  that  could  perform  the  work  assigned  to  IDA  as  effectively  and  efficiently.”  The  use  of 
for-profit  contracts  was  dismissed  as  “constrained  by  conflict  of  interest  concerns,  problems  in 
handling  data  proprietary  to  other  contractors,  and  other  competitive  sensitive  issues.” 

The  Review  notes  that  each  task  proposed  for  execution  by  IDA  is  evaluated  to  determine  if  IDA 
is  the  most  effective  source.  The  Review  reports  that  OSD,  the  Joint  Staff,  and  the  various 
Defense  Agencies,  all  have  omnibus  contracts  with  for-profit  firms  that  might  be  available  for  use 
as  alternatives  to  IDA. 

IDA’s  cost  and  efficiency  were  evaluated.  The  Review  reported  that  more  than  300  individual 
tasks  are  accomplished  by  the  IDA  S&A  FFRDC  each  year,  and  that  IDA’s  management  is  taking 
corrective  action  to  reduce  unsatisfactory  results  in  the  limited  cases  where  they  occur.  The 
review  noted  that  some  task  sponsors  believed  the  cost  of  IDA  work  appear  high  compared  to 
some  for-profit  contractors.  The  review,  however,  concluded  that  “based  on  available  data,  IDA 
costs  are  lower  than  many  other  FFRDCs  and  are  comparable  with  those  of  for-profit  contractors 
with  similar  work  force  composition  and  other  relevant  characteristics.”  The  average  MTS  cost 
at  IDA  was  given  as  approximately  S124K,  that  for  RAND  NDRI  was  estimated  at  $136K. 
Defense  Supply  Service— Washington  reportedly  showed  equivalent  contractor  rates  of  S126K  for 
senior  systems  analysts  for  one  contractor  and  $156K  for  a  senior  operators  research  analyst  from 
a  second  contractor. 

The  Review  reported  that  the  majority  of  task  sponsors  agreed  that  IDA’s  management  is 
responsive  to  the  needs  of  DoD  and  that  the  DCAA  reviews  between  October  1989  and 
December  1992  revealed  no  major  problem  with  IDA’s  internal  controls.  The  Review 
recommended  approval  of  a  continuation  of  the  contract. 
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Communications  and  Computing  FFRDC 

Much  of  the  material  provided  for  the  Communications  and  Computing  FFRDC  was  classified. 
The  material  presented  here  is  an  unclassified  summary. 

Objectives  and  guidelines  for  scope  of  work,  organizational  structure,  size,  appropriateness  of 
customers 

The  Center  is  sponsored  by  the  National  Security  Agency.  It  conducts  fundamental  research  for 
the  NS  A  in  (1)  cryptology,  including  the  creation  and  analysis  of  complex  encipherment 
algorithms,  as  well  as  in  speech  and  signal  analysis;  and  (2)  various  technologies  associated  with 
supercomputing  and  parallel  processing  including  new  architectures,  hardware,  and  software 
(including  prototypes),  as  well  as  parallel  processing  algorithms  and  applications. 

The  Center  serves  a  single  sponsor.  It  works  in  problem  areas  identified  by  NSA.  The  structure 
of  the  Center  is  shown  in  figure  C-l  above. 

DoD  funding  for  the  Center  is  shown  in  table  C-4. 


Table  C-4  :  DoD  Funding 

FY  1991  FY  1992  FY  1993  FY  1994  FY  1995 

IDA-Communications  and 

Computing  32.5  35.0  32.6  34.1  32.6 

SOURCE:  Department  of  Defense 

There  were  142  MTS  involved  in  the  Center  in  FY  95.  The  Management  Plan  projects  133.5 
STE  for  FY  96  and  1 50  for  FY  98  and  beyond. 

The  composition  of  the  staff  is  shown  in  figure  C-8. 
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Figure  C-8  :  Composition  of  IDA  Communications  and  Computing  FFRDC 

Research  Staff 

PhDs  78%,  Masters  12% 


SOURCE:  Institute  for  Defense  Analyses 
The  core  competencies  of  the  Center  are: 

•  Cryptologic  Mathematics 

•  Computing  Sciences 

•  Communications  Theory 

The  level  and  appropriateness  ofnon-DoD  work  and  work  for  other  than  the  primary  sponsor 

The  Center  conducts  no  non-core  work  and  although  the  sponsoring  agreement  permits  it  to  do 
so,  conducts  no  work  for  anyone  other  than  NS  A. 

Compliance  with  the  DoD  Management  Plan 

The  DoD  plans  directs  Research  and  Development  (R&D)  Laboratories  to  maintain  expertise  and 
related  competencies  necessary  to  address  the  core  work  and  priorities  of  the  sponsor. 

More  generally,  the  Centers  are  directed  to  perform  only  core  work.  Such  work  must  be 
approved  by  the  primary  sponsor,  can  only  be  accomplished  for  DoD,  other  government  entities 
(including  state  and  municipal  governments),  and  for  not-for-profit  activities;  and  cannot  include 
commercial  work. 
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Parent  institutions  operating  DoD  FFRDCs  may  perform  non-FFRDC  work,  but  such  work  must 
conform  to  some  specific  guidelines:  be  in  the  national  interest;  not  undermine  the  independence, 
objectivity  and  credibility  of  the  institute  or  call  into  question  the  freedom  from  conflict  of 
interest;  not  take  unfair  advantage  of  FFRDC  work;  and  normally  work  in  the  public  sector, 
although  commercial  work  may  be  accepted  if  the  sponsor  agrees. 

Fees  may  be  authorized.  The  Plan  calls  for  “an  annual  fee  proposal  justifying  each  element  of 
fee.” 

Sponsors  are  directed  to  ensure  the  FFRDC  is  used  only  for  intended  purposes,  and  that  the  costs 
are  reasonable  and  products  are  of  high  value,  sponsors  review  work  to  ensure  it  is  appropriate  to 
be  done  and  they  assure  DDR&E  that  all  directives  are  carried  out. 

The  sponsor  and  IDA  report  that  the  center  conducts  only  core  work.  IDA  conducts  no  non- 
FFRDC  work. 

NS  A  states  it  is  aware  of  the  way  fees  are  used. 

NS  A  uses  the  Five-Year  Review  to  judge  the  work.  It  reports  that  it  has  concluded  that  EDA 
provides  an  outstanding  research  program  in  return  for  NSA’s  investment. 

Sponsors’  Management  Process 

The  NSA  reports  that  IDA’s  efficiency  and  cost-effectiveness  as  a  contractor  are  assessed  by 
reviewing  technical,  management,  and  financial  reports  submitted  pursuant  to  the  contract. 

The  sponsor  reported  that  the  mandated  ceilings  have  disrupted  planning  processes. 

Thoroughness  of  the  Five-Year  Review 

The  contract  renewal  process  reportedly  adheres  to  the  rules  of  such  processes.  The  sponsor 
reports  that  a  comprehensive  review  is  completed  and  an  exhaustive  CICA  report  is  prepared. 
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Appendix  D 


Logistics  Management  Institute 

The  written  and  oral  information  presented  to  the  Task  Force  by  both  the  Logistics  Management 
Institute  (LMI)  and  its  primary  sponsor  is  summarized  in  this  appendix.  The  OSD  sponsor  was 
very  supportive  of  the  need  for  the  work  conducted  by  LMI  and  the  value  of  LMI  products.  The 
Sponsor  noted  the  wide  range  of  work  conducted  by  LMI  and  the  critical  importance  of  the  topics 
LMI  addresses.  The  Sponsor’s  view  is  that  “LMI  is  critical  to  modernizing  logistics  by  the  21st 
century.” 

Objective  and  guidelines  for  scope  of  work,  organizational  structure,  size,  and  appropriateness 
of  customers. 

Logistics  Management  Institute  was  established  in  1961  as  a  nonprofit  corporation  at  the  request 
of  Defense  Secretary  Robert  N.  McNamara  to  gather  an  expert  staff  to  bring  improved  business 
practices  to  the  Department’s  logistics  operations  and  to  serve  the  long-range  analytical  needs  of 
the  Department.  Between  1961  and  1984  the  Institute  operated  as  a  non-profit  entity  with  a 
series  of  5-year  sole-source  contracts.  In  1983  the  Department  decided  to  establish  an  FFRDC 
for  logistics  research  and  analyses.  In  1984,  Logistics  Management  Institute  was  selected  to 
operate  an  (FFRDC)  to  perform  research,  studies,  and  analyses  to  support  the  Office  of  the 
Secretary  of  Defense  (OSD),  the  Joint  Staff  (JS),  Defense  Agencies,  the  Military  Departments  and 
Unified  and  Specified  Commands.  The  primary  sponsor  was  the  Assistant  Secretary  of  Defense 
(Manpower,  Installations  and  Logistics)  and  successor  organizations.  In  1995,  the  Under 
Secretary  of  Defense  (Acquisition  &  Technology)  was  named  LMTs  primary  sponsor  with  the 
Deputy  Under  Secretary  of  Defense  (Logistics)  as  the  Executive  Agent. 

Logistics  Management  Institute  is  structured  into  two  centers,  the  DoD  FFRDC  (LMI),  and  a 
Center  for  Public  Administration  (CPA).  The  organizational  structure  and  the  criteria  used  for 
selecting  work  in  both  these  centers  is  shown  in  figure  D-l . 

The  LMI  FFRDC  works  for  its  DoD  and  non-DoD  clients  under  all  the  rules  governing  FFRDCs. 
The  Center  for  Public  Administration  (CPA),  established  in  1994,  works  for  both  government  and 
nonprofit  clients  (it  conducts  no  work  for  the  DoD)  and  operates  under  a  full  disclosure 
arrangement  with  the  DoD.  The  CPA  makes  up  only  a  small  portion  of  the  Institute’s  work. 

Total  revenue  for  FY  1995,  and  its  distribution,  is  shown  in  figure  D-2. 
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Figure  D-l:  Logistics  Management  Institute 


Criteria: 

-  Core  competency 

-  FFRDC  stability 

-  DoD  approved 


Criteria: 

-  Government  or  Nonprofit  clients 

-  Full  disclosure  to  DoD 

-  No  conflict  with  DoD  program 
•  No  unauthorized  use  of 

proprietary  or  priviliged 
information  from  LMI  FFRDC 


SOURCE:  LMI 


Figure  D-2:  LMI  FY  95  Revenue/Distribution 


Logistics  Management  Institute  LMI  FFRDC 


SOURCE:  LMI 


Recent  funding  trends  are  shown  in  figure  D-3. 
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Figure  D-3:  LMI  Funding  Profile  Actual  Obligations 
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SOURCE:  Department  of  Defense 

According  to  the  latest  Five-Year  Comprehensive  Review,  LMTs  primary  mission  is  to  provide 
the  DoD  leadership  with  expert  information,  and  independent,  objective  analysis  on  a  wide  range 
of  logistics,  acquisition,  and  related  infrastructure  issues  and  policy  initiatives. 

To  satisfy  DoD’s  needs,  LMI  states  that  it  has  developed  and  maintains  core  competencies  in  five 
areas: 

Acquisition:  Includes  assessment  of  procurement  policies,  identification  of  best  contracting 
practices,  evaluation  of  the  effects  of  policy  changes  on  small  businesses,  analysis  of  industrial 
base  issues,  examination  of  the  support  implications  of  new  procurements,  and  review  of 
manufacturing  technology  strategies. 

Facilities  and  Environment:  Evaluation  of  economic  impacts  of  realigning  government 
facilities,  assessment  of  construction  alternatives,  formulation  of  joint  public  and  private  financing 
projects,  and  development  of  environmental  strategic  plans. 

Force  Management:  Encompasses  development  of  decision  support  and  unit  readiness 
information  systems,  evaluation  of  force  mix  alternatives,  and  analysis  and  implementation  of 
business  process  improvements  in  civilian  personnel  management. 

Materiel  Management:  Includes  review  of  inventory  policies,  assessment  of  medical  distribution 
strategies,  evaluation  of  equipment  maintenance  alternatives,  reengineering  of  transportation 
management  processes,  development  of  logistics  models,  and  enhancement  of  logistics  systems. 


D-3 


Operational  Logistics:  Focuses  on  the  analysis  of  strategic  mobility  programs,  evaluation  of 
new  technologies  for  strategic  mobility,  assessment  of  wartime  logistics  alternatives,  development 
of  guidelines  for  battle  damage  repair,  and  formulation  of  logistics  options  for  NATO. 

Like  all  of  the  FFRDCs,  LMI  states  that  core  work  is  defined  by  the  combination  of  LMI’s 
mission,  purpose,  and  capabilities;  LMI’s  core  competencies;  and  the  FFRDC  strategic 
relationship  maintained  between  LMI  and  its  sponsors.  LMI  reports  that  non-core  studies  and 
analyses  work  will  not  be  done  by  the  FFRDC.  The  criteria  used  to  determine  if  a  task  is 
appropriate  core  work  for  the  FFRDC  are: 

•  Whether  the  work  is  consistent  with  LMI’s  mission,  purpose  and  capabilities: 

-  Responsive  to  the  most  important  needs  and  concerns  of  senior  DoD 
and  other  government  officials. 

-  Meets  special  technical  and  analytic  needs  that  cannot  be  met  as 
effectively  by  other  means. 

-  Requires  an  in-depth  understanding  of  logistics  and  acquisition  issues 
across  the  Department. 

-  Must  be  sponsored  by  an  appropriate  organization. 

-  Must  be  performed  effectively  using  the  capabilities  that  LMI 
maintains,  as  described  in  detail  above. 

•  Whether  it  requires  at  least  one  of  the  FFRDC ’s  five  core  competencies  that 
were  outlined  above; 

•  Whether  it  requires  one  or  more  of  the  characteristics  of  a  strategic  relationship 
between  the  DoD  and  LMI: 

-  Effective  performance  of  objective,  high  quality  work  on  subjects 
integral  to  the  mission  and  operations  of  sponsoring  offices. 

-  Freedom  from  real  and  perceived  conflicts  of  interest. 

-  Broad  access  to  information,  including  sensitive  government 
information,  proprietary  data  from  industry,  and  other  information  not 
normally  available  outside  the  government. 

-  Comprehensive  knowledge  of  sponsor  needs,  problems,  and  issues. 

-  Responsiveness  to  emerging  and  evolving  needs  of  sponsors. 

-  Long-term  continuity  of  knowledge  on  issues  and  problems  of 
enduring  concern,  including  both  maintaining  corporate  memory  for 
sponsors  when  appropriate  and  responding  to  quick-response  sponsor 
needs  in  areas  of  established  expertise. 

In  its  briefing,  the  sponsor  stated  that  the  LMI  work  is  needed.  The  sponsor  also  stated  that  the 
FFRDC  works  on  ill-defined  important  issues  that  are  not  amenable  to  well  defined  Statement  of 
Work  allowing  for  meaningful  competition. 
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The  FFRDC  may  work  for  Government  Departments  and  Agencies  other  than  the  Department  of 
Defense  in  areas  directly  related  to  the  core  areas  described  in  this  document,  subject  to  the 
review  and  approval  process  described  in  LMTs  sponsoring  agreement  and  the  DoD  Management 
Plan.  The  FY  95  funding  breakdown  for  DoD  and  non-DoD  work  in  core  areas  is  shown  in  table 
D-l. 


The  FFRDC’ s  parent  organization,  the  Logistics  Management  Institute  and  its  non-FFRDC 
division  the  Center  for  Public  Administration,  may  perform  work  for  other  organizations  subject 
to  applicable  laws  and  regulations,  and  subject  to  the  restrictions  imposed  by  the  DoD  to  protect 
the  strategic  relationship  necessary  to  the  operation  of  the  FFRDC. 

LMI  reports  that  as  a  matter  of  policy,  the  Institute  shall  accept  only  work  that  is  in  the  public 
interest,  fully  disclose  non-FFRDC  work  to  the  DoD,  and  periodically  review  with  DoD  the 
appropriateness  of  the  non-FFRDC  work. 

Table  D-l:  LMI  FY  95  Funding  Overview 

($  Millions) 


Core  Competency 

OSD 

Military 

Departments 

Defense 

Agencies 

JCS& 

CINCS 

Non-DoD 

Total 

Materiel  Management 

3.0** 

2.5 

2.6 

0.9 

4.3 

13.3 

Operational  Logistics 

0.1 

3.5 

0.7 

1.2 

0.1 

5.6 

Acquisition 

3.0 

0.3 

1.4 

- 

2.0 

6.7 

Installations  &  Environment 

0.9 

2.5 

0.2 

- 

2.3 

5.9 

Force  Management 

6.6* 

1.6 

- 

- 

- 

8.2 

Total  13.6*  10.4 

Executive  Agent,  DUSD(L)  is  largest  single  sponsor  @  $2.3. 

4.9 

2.1 

8.7 

39.7* 

SOURCE:  Department  of  Defense _ 

The  Logistics  Management  Institute  reports  that  it  abides  by  the  following  restrictions  on 
acceptance  and  performance  of  non-FFRDC  work: 

•  Non-FFRDC  work  shall  be  free  of  conflict  of  interest  with  work  performed  for 
DoD. 

•  No  unauthorized  use  shall  be  made  of  proprietary  or  privileged  information 
gained  from  activities  of  the  FFRDC. 

•  Acceptance  and  performance  of  non-FFRDC  work  shall  not  jeopardize  the 
performance  of  work  by  the  FFRDC. 
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•  Performance  shall  be  conducted  in  accordance  with  applicable  cost  accounting 
standards  and  accounting  principles  and  will  not  cause  an  issue  of  non- 
compliance  with  government  cost  principles  employed  under  the  FFRDC. 

Like  the  other  FFRDC  s,  LMI  reports  that  its  strategic  relationship  to  its  sponsor  is  characterized 

by: 


•  Objective  high-quality  work 

•  Freedom  for  real  and  perceived  conflict  of  interest 

•  Broad  access  to  information 

•  Comprehensive  knowledge  of  sponsor  needs  and  problems 

•  Long-term  continuity 

LMI  also  adds  that  two  other  qualities  are  necessary: 

•  Flexibility 

•  Adaptability 

LMI  has  a  staff  of  21 5  professionals  (205  working  for  the  FFRDC),  and  a  68-member 
administrative  staff  Eighty  percent  of  the  staff  hold  advanced  degrees;  most  in  business, 
economics,  engineering,  and  mathematics.  Ten  percent  hold  doctorates.  The  average  research 
fellow  has  21  years  of  professional  experience.  Staffing  trends  are  shown  in  figure  D-4. 


Figure  D-4:  Staff  Trends 


Year 


SOURCE:  LMI 

LMI  reports  that  the  growth  in  staff  is  in  direct  response  to  increased  DoD  need,  rather  than  any 
plan  for  growth.  The  Primary  Sponsor  reported  that  the  demand  for  LMI  is  growing  in  response 
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to  continued  global  commitments,  tight  budgets,  and  the  downsizing  and  restructuring  occurring 
within  both  the  government  and  in  defense  industry.  LMI  reported  it  turned  away  $12  to  SI 5 
million  in  core  work  in  FY  95  as  a  result  of  the  congressionally  mandated  ceilings.  The  Sponsor 
stated  that  LMI,  because  of  its  unique  long-term  relationship  and  experience  of  its  staff,  conducts 
work  that  cannot  be  conducted  elsewhere. 

The  level  and  appropriateness  of  non-DoD  work  and  work  for  other  than  the  primary  sponsor. 

As  noted  above,  LMI  engages  in  non-DoD  work  in  both  its  FFRDC  and  in  the  CP  A.  In  FY  95, 
$8.7  million,  about  22  percent  of  total  FFRDC  funding  came  from  non-DoD  sources.  LMI  states 
that  all  of  the  work  done  within  the  FFRDC  is  core  work — whether  for  DoD  or  non-DoD 
sources.  Work  for  others  within  the  FFRDC  must  be  approved  by  the  sponsor.  So  far  in  FY  96, 
there  have  been  more  than  25  different  customers  (DoD  and  non-DoD)  in  the  FFRDC.  Non-DoD 
customers  include:  GSA,  CIA  Department  of  Commerce,  NASA  Postal  Service,  Veterans 
Administration,  and  FEMA.  As  noted  above,  work  done  in  the  CPA  is  also  subject  to  special 
rules  in  that  it  is  limited  to  government  or  nonprofit  clients.  The  work  is  also  subject  to  full 
disclosure  to  DoD  and  cannot  conflict  with  the  DoD  program.  Further,  LMI  works  to  ensure  that 
there  is  no  unauthorized  use  of  proprietary  or  privileged  information  from  the  LMI  FFRDC. 

Compliance  with  the  DoD  Management  Plan 

The  DoD  Management  Plan  directs  that  the  S&A  FFRDCs  (1)  maintain  a  relatively  stable  level- 
of-effort,  and  (2)  maintain  competency  in  core  areas.  Staff  years  of  technical  effort  (STEs)  are 
used  in  sizing  and  managing  DoD-funded  FFRDC  work. 

The  Plan  states  that  DoD  FFRDCs  may  only  perform  core  work,  that  such  work:  must  be 
approved  by  the  primary  sponsor;  can  only  be  accomplished  for  DoD,  other  government  entities 
(including  state  and  municipal  governments),  and  for  not-for-profit  activities;  and  cannot  include 
commercial  work. 

Parent  institution  operating  DoD  FFRDCs  may  perform  non-FRDC  work,  but  such  work  must 
conform  to  some  specific  guidelines:  be  in  the  national  interest;  not  undermine  the  independence, 
objectivity  and  credibility  of  the  institute  or  call  into  question  the  freedom  from  conflict  of 
interest;  not  take  unfair  advantage  of  FFRDC  work;  and  normally  work  in  the  public  sector, 
although  commercial  work  may  be  accepted  if  the  sponsor  agrees. 

Fees  may  be  authorized.  The  Plan  calls  for  “an  annual  fee  proposal  justifying  each  element  of 
fee.” 

LMI  reports  that  it  has  an  experienced  staff  of  professionals  with  an  average  of  21  years  of 
experience.  The  Institute  states  that  it  has  focused  on  core  competence  for  the  last  six  years. 

Staff  levels,  as  shown  in  figure  D-4,  have  more  than  doubled  over  the  last  seven  years. 

As  already  noted,  LMI  reported  that  it  complies  with  the  DoD  policy  on  DoD  and  non-DoD 
work,  and  has  procedures  to  ensure  that  only  core  work  is  done  in  its  FFRDC  and  that  non-DoD, 
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non-core  work  conducted  in  the  CPA  conforms  to  the  requirements  stated  in  the  DoD 
Management  Plan. 

The  DoD  Management  Plan  authorizes  the  use  of  fees  and  LMI’s  sponsor  concluded  a  fee  was 
appropriate  in  its  most  recent  5-year  contract.  LMI  received  a  fee  of  4.3  percent  of  cost.  The 
Institute  has  not  been  contacted  about  the  need  for  an  annual  fee  proposal  justifying  the  fee.  The 
Institute  is  on  record  with  DDR&E  that  our  current  contract  to  operate  the  FFRDC  provides  for  a 
fixed  fee  for  the  five-year  period.  LMI  trusts  the  contracting  officer  will  honor  the  provision  of 
the  contract. 

Sponsor’s  Management  Process 

LMI’s  primary  sponsor  is  the  Under  Secretary  of  Defense  (Acquisition  and  Technology).  The 
Executive  Agent  for  LMI  is  the  Deputy  Under  Secretary  of  Defense,  Logistics.  The  contracting 
activity  is  the  Army  Defense  Supply  Service,  Washington  (DSS-W). 

Because  of  the  subject  it  works  on,  LMI  has  a  large  number  of  customers.  The  Executive  Agent 
reports  that  individual  sponsors  evaluate  work,  and  that  an  annual  review  is  conducted  by 
numerous  sponsors.  Each  project  description  contains  a  background  statement,  a  scope  of  work, 
a  period  of  performance,  reporting  requirements,  an  estimated  level  of  effort  as  well  as  the  name 
of  a  designated  project  monitor.  The  Executive  Agent  reports  that  his  office  performs  a  final 
review  of  all  work  assigned  to  LMI  according  to  the  following  criteria: 

•  That  the  task  described  in  the  project  statement  of  work  meets  the  “Purpose, 

Mission  and  General  Scope  of  Effort”  criteria  as  defined  in  LMI’s  Sponsoring 
Agreement, 

•  That  LMI  has  the  ability  to  satisfy  the  DoD  requirement  for  the  project/study, 

•  That  no  other  known  contractors  have  the  experienced  personnel,  technical 
expertise,  absent  of  conflict,  objectivity,  and  capability  necessary  to  perform 
this  effort  as  effectively  as  LMI, 

•  That  the  effort  fully  complies  with  applicable  procurement  statutes,  policies, 
and  regulations  for  non-competitive  actions,  and 

•  That  the  effort  is  not  being  undertaken  at  LMI  to  avoid  competition. 

The  Executive  Agent  also  reports  that  the  LMI  effort  is  fully  reviewed  as  a  part  of  the 
Comprehensive  Service  Process  and  as  a  result  of  IG  review  and  routine  and  special  audits.  The 
latest  Comprehensive  Review  reported  that: 

Each  sponsoring  organization  has  its  own  review  process  to  assure  that  the  work  is 
appropriate  for  assignment  to  LMI.  Task  sponsors  assured,  in  the  survey,  that 
they  are  knowledgeable  of  the  market  place  regarding  the  level  of  support  these 
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alternative  sources  can  provide  in  the  studies  and  analyses  of  logistics  and 
acquisition. 

The  most  recent  Comprehensive  Review  included  a  survey  of  task  sponsors  and  project  monitors, 
and  18  senior  DoD  officials  who  were  queried  about  the  value  of  LMI  work.  Responses  were 
reportedly  very  favorable. 

LMI  and  its  Primary  Sponsor  report  a  number  of  management  controls  are  in  place  to  ensure  the 
cost  effective  operation  of  LMI.  They  include  oversight  by  the  primary  sponsor,  DoD  regulatory 
organizations,  and  the  LMI  management.  There  is  also  review  by:  the  contracting  office,  DSS- 
W,  the  Defense  Contract  Audit  Agency  (DCAA),  and  the  Defense  Contract  Management  Area 

Office  (DCMAO).  The  Institute’s  management  emphasizes  its  own  internal  reviews.  LMI 

employs  the  services  of  Deloitte  &  Touche  for  accounting  services.  The  Institute  s  Trustees  also 
have  an  Audit  Committee  and  Compensation  Committee  who  have  oversight  of  accounting, 
personnel,  and  cost  issues. 

OUSD(A&T)  is  responsible  for  contract  oversight  of  LMI  and  reports  that  LMI  research  is 
monitored  and  evaluated  at  the  project  and  sponsor  levels.  LMI  provides  monthly  cost  reports  on 
each  project. 

The  DoD/LMI  Advisory  Board  is  responsible  for  ensuring  that  the  FFRDC  is  being  properly 
utilized  The  Board  approves  the  overall  research  agenda  and  addresses  broad  substantive  and 
FFRDC  policy  concerns. 

The  Five-Year  Review  reports  that  the  Administrative  Contracting  Officer  (ACO)  and  the 
Defense  Contract  Audit  Agency  (DCAA)  review  the  Institute’s  cost  accounting  and  management 
systems.  According  to  the  last  Comprehensive  Review,  the  Defense  Contract  Audit  Agency  has 
found  no  material  deficiencies  in  the  Institute’s  accounting  practices,  the  Institute  is  fully 
compliant  with  Cost  Accounting  Standards,  and  has  operated  within  the  guidelines  detailed  in  its 
disclosure  statement.  Indirect  rates  are  reviewed/adjusted  and  approved  by  DCAA  annually. 
Costs  on  specific  projects  related  to  labor-hours,  travel  and  sometimes  equipment,  are  negotiated 
with  the  Defense  Supply  Service,  Washington  (DSS-W)  contracting  office  at  the  time  the  task 
order  is  negotiated. 

The  Institute  reports  that  it  continuously  seeks  to  improve  its  internal  processes  and  has 
maintained  relatively  constant  indirect  rates  over  the  past  four  years.  It  states  that  these  rates  are 
in  line  with  those  of  most  reputable  commercial  studies  and  analyses  firms.  The  1995  Five  Year 
Review  reported  that  the  Member  of  the  Technical  Staff  (MTS)  costs  for  LMI  are  lower  than 
those  of  any  other  FFRDC.  The  Institute  reported  it  had  recently  abolished  two  senior  indirect 
positions.  Project  leaders  get  cost  information  twice  each  month.  Non-labor  costs  are  regularly 
entered  and  project  leaders  have  that  information  via  electronic  cost  reports  available  daily. 
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The  Thoroughness  of  the  Five-Year  Review 

The  latest  Comprehensive  Review  of  LMI  is  dated  30  March  1995.  The  Review  addresses  most 
of  the  points  required  in  the  new  DoD  Management  Plan  and  generally  covers  the  topics  required 
by  OFPP  Policy  Letter  84-1  and  the  FAR.  It  provides  some  discussion  of  the  sponsor’s  and 
DoD’s  needs  for  LMI’s  work.  Relatively  detailed  examples  of  work  and  results  were  provided 
for  each  of  the  core  areas.  The  Review  has  a  very  brief  and  very  general  discussion  of  alternative 
sources  that  might  meet  the  sponsor’s  needs.  Alternatives  are  discussed,  and  dismissed,  in  very 
general  terms. 

An  evaluation  of  the  efficiency  and  effectiveness  of  LMI  is  partially  included  in  the  detailed 
examples  of  work  done  and  the  impact  of  that  work.  In  addition,  the  sponsor  reported  a  survey 
of  task  sponsors,  project  monitors,  and  senior  DoD  personnel,  all  of  who  reportedly  express 
strong  satisfaction  with  LMI’s  support.  The  Review  complies  with  the  earlier  requirement  for 
evaluation  of  the  quality  and  cost  of  LMI’s  work,  but  it  does  not  appear  to  fully  comply  with  the 
New  Management  Plan’s  level  of  desired  analysis  reporting  “the  number  and  dollar  value  of 
projects  and  programs  assessed.” 

Management  controls  to  ensure  a  cost-effective  operation  are  discussed  only  in  general  terms  and 
appear  to  be  based  more  on  assurance  of  proper  cost  accounting  (compliance  with  DCAA  rules), 
than  with  a  careful  comparison  of  costs.  The  Review  does  report  that  LMI  MTS  cost  is  the 
lowest  of  any  of  the  FFRDCs. 
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Appendix  E 


The  RAND  Corporation 

The  RAND  Corporation  is  a  private,  non-profit  corporation  with  headquarters  in  Santa  Monica, 
CA.  RAND  was  incorporated  in  May  1948,  with  initial  capital  donated  by  The  Ford  Foundation, 
after  a  two-year  gestation  period  as  Project  RAND  conducted  under  a  special  and  experimental 
contract  between  the  Army  Air  Forces  and  Douglas  Aircraft  Company.  In  1950,  when  RAND 
began  expanding  from  its  Project  RAND  contract,  it  established,  at  the  suggestion  of  the  Air 
Force,  a  smaller  contract  with  the  Atomic  Energy  Commission.  RAND  also  began  work  for  OSD 
in  the  late  1950s.  In  1959,  RAND  began  work  with  ARPA  and  NASA.  At  that  point,  the  Air 
Force  announced  that  it  intended  to  freeze  its  support  of  RAND  at  its  then  current  dollar  level. 

In  1961  RAND  added  the  National  Science  Foundation  and  the  National  Institutes  of  Health  to  its 
clients.  As  the  decade  of  the  1960s  continued,  RAND  worked  for  OSD,  various  defense 
agencies,  and  the  Agency  for  International  Development.  In  addition,  it  received  grants  and 
contracts  from  other  federal  agencies,  non-profit  organizations,  and  the  City  of  New  York  for 
studies  on  a  wide  range  of  social,  economic,  and  methodological  issues  (including  transportation, 
housing,  health  care,  communications,  urban  development,  and  others).  Project  RAND,  funded 
by  the  Air  Force,  shifted  during  this  period  from  being  95  percent  of  RAND’s  budget  to  68 
percent:  RAND  had  over  a  dozen  different  clients. 

By  1976,  RAND’s  research  for  the  Department  of  Defense  accounted  for  only  about  half  of  the 
institution’s  total  revenue.  In  that  year,  the  DoD  limited  the  FFRDC  designation  to  RAND’s  Air  Force 
research  contract,  which  was  renamed.  Project  AIR  FORCE.  RAND’s  other  research  for 
DoD — performed  mainly  for  OSD  and  defense  agencies — was  thereafter  conducted  under  several 
separate  long-term  contracts.  In  1984,  that  research  was  consolidated  by  DoD  into  one  contract,  given 
an  FFRDC  designation,  and  named  the  National  Defense  Research  Institute  (NDRI).  DoD  also 
established  new  oversight  arrangements  for  that  work,  including  a  new  senior  advisory  group.  Around 
the  same  time,  the  Army  transferred  its  FFRDC,  the  Arroyo  Center,  from  the  Jet  Propulsion 
Laboratory  to  RAND.  In  1 992,  RAND  won  a  competition  for  a  fourth  FFRDC,  the  Critical 
Technologies  Institute,  which  was  created  by  Congress  to  serve  the  Office  of  Defense  Technology 
Policy.  In  addition  to  the  research  within  the  FFRDCs,  RAND  conducts  policy  research  for  other 
government  agencies,  private  foundations,  and  donors  through  its  other  institutes  and  programs. 
RAND  does  no  research  that  will  become  proprietary  to  private  sources.  RAND  also  operates  a  fully 
accredited  Ph  D.  program.  The  RAND  Graduate  School,  which  was  established  in  1970  (see  figure  E- 
1  for  organization). 

Each  of-RAND’s  DoD  FFRDCs  has  its  own  sponsor,  contracting  arrangement,  and  charter.  Each 
is  supported  by  a  five-year  contract.  Project  AIR  FORCE  and  the  Arroyo  Center  receive  line- 
item  fimding  from  their  Service  sponsors.  The  NDRI  receives  funds  on  a  project  basis  from  its 
primary  sponsor,  the  Office  of  the  Secretary  of  Defense,  and  from  the  Joint  Staff. 


Figure  E-l:  The  RAND  Corporation 


SOURCE:  RAND  Corporation 

Each  RAND  FFRDC  is  overseen  by  an  advisory  group  that  determines  the  genera]  direction  of 
that  FFRDC’ s  research,  reviews  and  evaluates  its  products,  and  may  help  set  its  budget  level.1 
For  Project  AIR  FORCE  and  the  Arroyo  Center,  the  majority  of  the  members  of  the  board  are 
military  officers  at  the  general  officer  level  with  some  senior  civilian  members.  In  1996,  the  board 
for  the  National  Defense  Research  Institute  was  composed  entirely  of  senior  civilian  members 
from  the  Office  of  the  Secretary  of  Defense. 

The  total  revenue  from  clients  reported  by  RAND  for  1995  was  $106.8  million.  Of  this,  86  percent 
came  from  government  sources,  both  federal  and  local,  10  percent  from  foundations,  and  4  percent 


1  Project  AIR  FORCE  budget  is  determined  by  the  POM/PB  process. 
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from  other  sources.  National  security  (DOD  and  the  intelligence  community)  made  up  67  percent  of 
RAND's  revenue. 

The  largest  segment  of  RAND's  non-DOD  work  is  in  the  area  of  Health  Sciences  (13%  of  total). 
RAND  also  conducts  studies  in  education,  labor  and  population,  technology  policy,  civil  justice, 
criminal  justice,  and  other  subjects. 

The  RAND  Corporation,  including  the  FFRDCs,  had  a  total  of  1,091  resident  staff  at  the  end  of  FY 
95.  RAND  reached  its  peak  employment  in  1957  when  it  had  2,605  people.  This  included  a 
significant  training  effort  for  the  Air  Defense  Directorate  that  was  later  spun  off  to  form  the 
Systems  Development  Corporation.  Fifty-eight  percent  of  the  FY  95  staff  were  professional,  24 
percent  research  support,  and  1 8  percent  general  support.  There  were  also  65  visiting  fellows 
(graduate  and  military)  and  both  resident  and  non-resident  consultants  contributing  a  total  of 
approximately  1 1 5  FTE  of  effort. 


Figure  E-2:  Composition  of  RAND  Research  Staff,  1995 


■ 

Doctorates  47%,  Masters  31%,  Bachelors  19%,  none  3% 


SOURCE:  RAND 

Overall,  the  technical  staff  is  highly  educated  (Figure  E-2).  Forty-seven  percent  have  PhD.  degrees, 
while  3 1  percent  have  Masters,  and  only  3  percent  having  no  degrees.  One  third  have  degrees  in 
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engineering,  mathematics,  operations  research,  or  one  of  the  sciences.  The  remainder  have  degrees  in 
social  sciences,  economics,  and  a  large  number  of  other  disciplines  and  fields.  RAND  has  a  matrix 
structure.  The  staff  are  not  assigned  to  a  particular  division  or  institute,  but  may  work  across  them. 

The  manpower  numbers  (MTS)  shown  in  the  RAND  DoD  FFRDCs  are  ceilings  that  can  be  filled  by 
personnel  from  the  entire  RAND  staff. 

The  overall  average  professional  salary  at  RAND  is  about  $76,500.  In  FY  95,  the  multiplier  was  1 .456 
for  “benefits”  (including  paid  leave)  and  1.794  for  “overhead,”  for  a  total  multiplier  of 2.628.  This  does 
not  include  fee;  the  RAND  fee  for  FY  95  ranged  from  5.74  to  6.00  percent.  The  total  fee  for  the  DoD 
FFRDCs  in  both  FY  94  and  FY  95  was  about  $4  million.  With  fee  added  in  this  would  produce  a 
multiplier  of 2.779  to  2,786  for  a  man-year  cost  of  $212,593  to  $213,129. 

Over  the  past  five  years  the  labor  costs  at  RAND  have  been  relatively  stable  while  benefits  and 
overhead  multipliers  have  grown  slightly. 

The  three  DoD  FFRDCs  are  discussed  in  detail  below. 
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Project  AIR  FORCE 


Objectives  and  guidelines  for  scope  of  work,  organizational  structure,  size,  and  appropriateness  of 
customers. 

Project  AIR  FORCE  as  noted  above,  is  the  direct  descendant  of  the  research  begun  by  the  Army  Air 
Forces  in  1 948  at  the  Douglas  Aircraft  Company.  That  work,  known  as  Project  RAND,  was 
transferred  to  the  independent,  non-profit  RAND  Corporation  when  it  was  established  in  1948. 

Project  RAND  was  renamed  Project  AIR  FORCE  in  1976  to  distinguish  the  FFRDC  contract  from  the 
rest  of  RAND. 

The  mission  of  PAF  as  stated  in  the  Sponsoring  Agreement  is  to: 

•  conduct  a  continuous  interrelated  program  of  objective  analysis  on 
major  cross-cutting  policy  and  management  issues  of  enduring  concern 
to  the  Ar  Force 

•  maintain  expertise  on,  and  an  institutional  memory  about,  Ar  Force 
operations  and  policy 

•  explore  long-term  issues  and  questions  that  will  become  vital  areas  of 
concern  in  the  future 

•  provide  timely  short-term  assistance  on  urgent  problems  consistent 
with  PAFs  demonstrated  expertise. 

The  Project  Ar  Force  (PAF)  organizational  structure  is  shown  in  figure  E-3. 

Figure  E-3:  Project  Air  Force  FFRDC 


SOURCE:  RAND 

PAF  core  capabilities  and  related  areas  of  expertise  are  shown  in  table  E-l 
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Table  E-l :  PAF  Core  Capabilities  and  Areas  of  Expertise 


Strategy  and  Doctrine 

Force  Development 
and 

Application 

Resource  Management 

Emerging  global  and  regional 

Application  of  new  technologies  and 

integration  of  sustainment  and 

threats  and  their  implications  for  Air 

concepts  of  operations  and  force 

support  activities  to  increase 

Force  operations 

modernization  strategies 

operational  capabilities 

•  regional  political  and  military 

♦  combat  aircraft 

•  logistics 

developments 

•  defense  strategy,  including  arms 

•  space-based  operations 

•  R&D  and  acquisition 

control  and  deterrence 

•  C4I,  surveillance,  targeting,  info 
warfare 

•  mobility 

•  weapons 

•  modeling  and  simulation 

•  industrial  base 

•  defense  planning 

Air  Force  and  joint  doctrine 

•  privatization/outsourcing 

•  manpower 

•  cost  analysis 

Base  Force  Analysis 

Desert  Shield/Desert  Storm 

Acquisition  Reform 

Asymmetric  Strategies 

Modernizing  Bomber  Force 

Lean  Logistics 

SOURCE:  RAND 


PAF  has  remained  relatively  stable  in  size,  even  during  the  defense  buildup  of  the  early  1980s. 
The  trends  in  PAF  size  are  shown  in  figure  E-4. 

Figure  E-4:  Historical  Level  of  Effort  (MTS)  for  PAF 


SOURCE:  RAND 
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In  conducting  integrated  research,  PAF  reports  that  it  forms  multi-disciplinary  study  teams  drawn 
from  nearly  600  RAND  analysts  representing  a  broad  range  of  academic  disciplines.  The  majority 
of  the  staff  are  located  in  RAND’s  Santa  Monica,  California  headquarters;  approximately  ten 
percent  are  housed  in  the  Washington  D  C.  office.  The  RAND  staff  is  supplemented  by  other 
professionals  acting  as  consultants  on  a  part-time  basis. 

PAF  briefing  noted  that  the  congressional  ceilings  had:  disrupted  the  Air  Force  plan  for  PAF 
level-of-effort;  had  curtailed  research  on  personnel  and  training,  the  international  security 
environment,  and  modeling  and  simulation;  and  had  diverted  management  time  for  research  to 
other  matters. 

Funding  is  shown  in  figure  E-5. 


Figure  E-5:  Project  Air  Force  Funding 


SOURCE:  U.S.  Air  Force 

Funding  for  Project  AIR  FORCE  comes  entirely  from  a  single  line  item  in  the  defense  budget.  For  the 
last  four  years  (FY  93  through  FY  96)  this  amount  has  been  $24  million.  Project  AIR  FORCE 
currently  has  a  ceiling  of  1 1 3  MTS,  along  with  appropriate  support  personnel.  For  FY  96,  a  Project 
AIR  FORCE  MTS  is  calculated  to  cost  $212,3892 

The  criteria  the  Air  Force  uses  for  measuring  the  appropriateness  of  work  to  be  placed  at  PAF  is 
shown  in  table  E-2. 


2  Calculated  from  the  current  level  of  effort,  with  a  $24  million  budget  supporting  113  MTS. 
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Table  E-2:  Criteria  for  Measuring  Appropriateness  of  PAF  Research 


Acceptance  Criteria 

Constraint  Criteria 

Does  the  proposed  research: 

Work  assignment  has  constraints.  It  is 
unacceptable  to  use  PAF: 

•  Fall  within  PAF’s  purpose,  mission,  scope,  or 
special  competency? 

•  Simply  because  proposed  research  cannot  be 
met  by  existing  alternatives 

•  Require  access  beyond  norm? 

•  as  convenience  for  work  others  could 
perform 

•  Require  objectivity,  independence,  freedom  from 
bias  or  conflict  of  interest? 

•  because  in-house  staff  is  fully  employed 

•  Help  to  maintain  research  balance? 

•  because  other  contract  vehicles  not  readily 
available 

•  Maintain  PAF  core  competencies  and/or  allow 
exploration  of  emerging  issues? 

•  PAF  cannot  compete  with  a  non-FFRDC 

•  Contribute  to  an  integrated  perspective  on  the 
issues 

SOURCE:  U.S.  Air  Force 


The  level  and  appropriateness  of  non-DoD  work  and  non-sponsored  work  undertaken  by  the 
FFRDCs  and  UARCs. 

The  sponsoring  agreement  states  that  PAF  may  accept  work  from  other  agencies  if  the  work  is 
determined  to  be  within  the  purpose,  mission,  and  general  scope  of  the  FFRDC,  and  is  consistent 
with  48  CFR  17.502  and  17.504(e),  and  is  specifically  authorized  by  the  Air  Force  sponsor. 

RAND  reports  that  the  only  current  customer  for  PAF  is  the  Air  Force.  The  PAF  briefing  noted 
that  it  can,  and  has,  accepted  non-sponsor  work  on  rare  occasions  (an  example  given  was  a  1989 
NASA  request  for  help  on  the  President’s  Space  Exploration  Initiative  which  was  approved  by  the 
Air  Force  Advisory  Group). 

Compliance  with  the  DoD  Management  Plan 

The  DoD  Management  Plan  states  that  annual  levels  of  effort  shall  be  based  on  application  of  the 
core  concept  and  for  S&A  FFRDCs,  guidelines  that  include:  (1)  maintaining  a  relatively  stable 
level-of-effort,  and  (2)  maintaining  competency  in  core  areas.  Staff  years  of  technical  effort 
(STEs)  are  used  in  sizing  and  managing  DoD-fimded  FFRDC  work. 

The  Plan  states  that  DoD  FFRDCs  may  only  perform  core  work,  that  such  work:  must  be 
approved  by  the  primary  sponsor;  can  only  be  accomplished  for  DoD,  other  government  entities 
(including  state  and  municipal  governments)  and  for  not-for-profit  activities;  cannot  include 
commercial  work. 

Parent  institutions  operating  DoD  FFRDCs  may  perform  non-FRDC  work,  but  such  work  must 
conform  to  some  specific  guidelines:  be  in  the  national  interest;  not  undermine  the  independence, 
objectivity  and  credibility  of  the  institute  or  call  into  question  the  freedom  from  conflict  of 
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interest;  not  take  unfair  advantage  of  FFRDC  work;  and  normally  work  in  the  public  sector, 
although  commercial  work  may  be  accepted  if  the  sponsor  agrees. 

Fees  may  be  authorized.  The  Plan  calls  for  “an  annual  fee  proposal  justifying  each  element  of 
fee.” 

As  figure  E-4  shows,  PAF  has  had  a  relatively  stable  workforce.  The  PAF  briefing,  however, 
reported  a  need  to  stabilize  at  a  “level  appropriate  to  mission  and  Air  Force  needs.  But  PAF  can 
also  call  on  assets  from  other  parts  of  RAND  to  meet  its  needs.  For  example,  the  Center  does 
collaborative  research  in  regional  sources  of  conflict,  logistics,  and  analytic  methods  and 
modeling.  PAF  management  stated  that  the  effective  size  of  PAF  is  shaped  by  the: 

•  range  of  critical  issues  confronting  senior  leadership  of  Air  Force 

•  depth  required  for  quality  and  timely  research,  especially  in  technical  areas 

•  breadth  of  PAF  mission  and  requirement  to  maintain  expertise  on  Air  Force  as  a  whole 

•  need  to  provide  research  assistance  on  time-urgent  problems 

The  goal  of  125  MTS  set  in  PAF’s  most  recent  Comprehensive  Review  reflected  these 
considerations.  Currently,  the  critical  mass  is  reportedly  being  eroded  by  the  reductions  in  both 
PAF  level  of  effort,  and  the  losses  in  the  pool  of  expertise  in  other  RAND  DoD  FFRDCs. 

PAF  states  that  it  focuses  on  core  work  requiring  its  unique  expertise.  It  does  not  undertake 
narrow  operational  or  technical  analyses,  nor  routine  support  activities.  It  also  does  not  conduct 
isolated  studies  outside  the  Center’s  areas  of  expertise,  or  provide  direct  staff  assistance. 

PAF  is  authorized  fees  in  its  sponsoring  agreement.  RAND  has  used  most  of  its  fee  for 
investments  in  its  human  capital;  exploratory  research,  extensions  and  derivations  of  sponsored 
projects,  and  staff  development.  RAND  management  expressed  concern  that  the  new 
management  guidelines  that  restructure  fee  determination  will  “eliminate  an  important  means  of 
rewarding  good  performance  and,  over  time, . .  .  .would  weaken  the  institutional  independence 
that  our  clients  have  come  to  value.”  The  Air  Force  and  OSD  disagree  with  this  view. 

Sponsor’s  Management  Processes 

The  Air  Force  sponsor  for  Project  Air  Force  is  the  Assistant  Secretary  of  the  Air  Force,  Acquisition. 
Oversight  is  provided  by  the  Air  Force  Advisory  Group  consisting  of  the  Vice  Chief  of  Staff  as  the 
Chairman  and  senior  military  and  civilian  personnel  in  the  Department  of  the  Air  Force  (see  figure  E- 
6).  The  panel  provides  general  policy  guidance,  establishes  major  research  issues,  approves  the  annual 
research  plan  and  reviews  research  outcomes.  The  panel  also  recommends  what  PAF’s  level-of-effort 
should  be  each  year. 
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Figure  E-6:  Air  Force  Advisory  Group 
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DCS/Logistics 

Director  of  Plans 
AFAG  Executive  Agent 


Principal  Deputy 
Assistant  Secretary  for 
Financial  Management 


Principal  Deputy 
Assistant  Secretary  for 
Acquisition 


DCS/Command, 
Control,  Communications, 
and  Computers 


SOURCE:  U.S.  Air  Force 


The  PAF  planning  process  is  outlined  in  figure  E-7. 
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Figure  E-7:  Annual  Research  Planning  Cycle  (FY) 
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SOURCE:  U.  S.  Air  Force 

The  Air  Force  oversight  process  is  illustrated  in  figure  E-8.  Metrics  used  in  the  annual  and  five-year 
reviews  are  shown  in  table  E-3. 
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Figure  E-8:  Air  Force  Management  Oversight 
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♦  Award  fee  review 

♦  Comprehensive 
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Feedback 


SOURCE:  U.S.  Air  Force 


Table  E-3  :  Metrics  Used  in  Annual  and  Five  Year  Reviews 


Annual  review 

--  technical  quality 

—  responsiveness 

—  value 

—  timeliness 

Award 

—  communication  (60%) 

(semi-annual) 

—  research  clarity  and  utility  (25%) 

-  efficiency/effectiveness  ( 1 5%) 

Comprehensive 

—  mission  needs 

Review  (every  five  years) 

-  alternative  sources 

—  effectiveness 

—  management  and  cost 

—  compliance 

The  Air  Force  reports  it  has  an  ongoing  process  of  both  internal  and  external  reviews  to  ensure 
that  PAF  is  a  cost  effective  operation.  This  process  involves  the  Procuring  Contract  Officer 
(PCO),  local  Administrative  Contracting  Officer  (ACO),  and  resident  Defense  Contract  Audit 
Agency  (DCAA)  auditor  performing  continuing  reviews  of  the  PAF  management  process.  The 
PCO  has  the  responsibility  to  ensure  that  sufficient  Air  Force  expertise  is  available  to  objectively 
evaluate  the  PAF  performance.  Further,  the  PCO  has  delegated  administration  of  the  contract  to 
the  ACO.  The  ACO  provides  assistance  in  the  performance  of  this  oversight  function  at  RAND 
Corporation  in  Santa  Monica  CA.  The  ACO  performs  contract  surveillance  and  verification  of 
costs  in  accordance  with  AFSCR  70-3,  Administration  of  Contracts  Awarded  to  Federally 
Funded  Research  and  Development  Centers.  AFSCR  70-3  provides  the  policy  pertaining  to  the 
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administration  of  FFRDC  contracts  and  details  the  responsibilities  of  the  ACO.  The  ACO  further 
relies  on  functional  area  experts  to  perform  verification  that  the  required  research  has  been 
performed.  One  of  the  resources  used  by  the  ACO  to  verify  the  accuracy  of  costs  claimed  is  the 
DCAA  auditor;  however,  the  ACO  retains  primary  responsibility  for  determining  that  the  costs 
claimed  by  PAF  are  reasonable  and  accurate.  Finally,  purchasing  systems  reviews,  compensation 
reviews,  overhead  audits,  budgetary  system  reviews,  billing  system  reviews,  internal  controls 
audits,  incurred  costs  audits,  Cost  Accounting  Standards  (CAS)  compliance  testing  audits, 
property  control  system  surveys,  annual  wage  and  salary  reviews,  estimating  systems  reviews,  and 
subcontract  reviews  are  periodically  conducted  by  resident  and  Defense  Contract  Management 
Agency  personnel  to  further  ensure  PAF's  cost  effectiveness. 

The  Thoroughness  of  the  Five-Year  Reviews 

The  latest  comprehensive  review  was  signed  in  November  1994.  The  review  examines  mission  needs, 
alternative  sources,  effectiveness,  management,  cost  and  compliance.  It  was  conducted  in  accordance 
with  OFPP  Policy  Letter  84-1  and  Federal  Acquisition  Regulations  35.017. 

The  Review  examined  the  Air  Force’s  special  technical  needs  and  mission  requirements  to  determine  if 
PAF  was  still  needed,  and  if  so,  at  what  level.  The  examination  reviewed  current  and  future  Air  Force 
needs,  and  stated  that  the  requirement  for  independent  support  from  an  organization  with  a  broad 
understanding  of  Air  Force  problems  was  as  great  as  ever.  The  Service  reported  that  an  independent 
research  agent  with  the  capabilities  of  an  FFRDC-like  organization  was  essential.  It  noted  that  in  1985 
4he  Air  Force  had  opened  the  RAND  contract  up  in  a  Commerce  Business  Daily  solicitation,  and  The 
fact  that  PAF  contract  was  awarded  to  the  RAND  Corporation  demonstrates  that  at  that  time  RAND 
was  the  best  qualified  source.” 

The  Review’s  consideration  of  alternative  sources  of  support  is  based  primarily  on  an  examination  of 
the  strengths  that  RAND’s  PAF  brings  to  the  table,  and  secondarily  on  an  examination  of  possible  in- 
house  and  private  sector  providers. 

The  Review  states  that  an  in-house  organization  would  probably  not  be  sufficiently  independent,  would 
have  difficulty  performing  cross-functional  interdisciplinary  research  and  developing  a  corporate 
memory.  These  factors  and  inadequate  compensation  prevent  in-house  provision  of  services;  conflict 
of  interest  problems  would  probably  eliminate  a  for-profit  firm;  and  the  cost  of  re-establishment  in 
another  not-for-profit  organization  makes  another  not-for-profit  an  unattractive  alternative.  There  are 
no  reported  specific  alternatives,  rather  the  alternatives  are  discussed  in  general  terms. 

PAF  efficiency  and  effectiveness  are  examined  in  terms  of  customer  interaction,  quality  of  research,  and 
management  effectiveness.  These  assessments  are  conduced  semiannually  by  polling  the  general  officer 
(or  SES)  project  sponsors.  They  are  asked  to  evaluate  PAF’s  performance  in  the  areas  of 
“Communications”  and  “Research  Clarity  and  Utility.”  The  PAF  program  office  evaluates  PAF  s 
management  effectiveness.  Evaluation  criteria  are  shown  in  box  E-l . 

The  assessment  of  FFRDC  management  to  ensure  cost-effective  operations  outlines  the  overall  Air 
Force  process  for  examining  costs.  The  Air  Force  reported  that  a  recent  DDR&E  evaluation 
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compared  MTS  rates  of  the  eleven  DoD  FFRDCs  with  three  representative  Contract  Advisory  and 
Assistance  Services  (CAAS)  contractor  labor  rates.  The  RAND  FFRDCs  (i.e.,  PAF,  Arroyo,  NDRI) 
reportedly  had  salary  rates  that  were  almost  4%  lower  than  the  next  lowest  salary  rate,  and  that  were 
over  34%  lower  than  the  highest  salary  rate.  The  total,  fully  loaded,  annual  MTS  labor  rate  for  the 
RAND  FFRDCs  were  reportedly  almost  3%  below  the  mean  of  the  rates  compared. 

The  Review  listed  a  number  of  steps  PAF  had  taken  to  upgrade  its  cost-effectiveness.  It  reorganized 
into  new  structure,  worked  closely  with  the  Air  Force  Sponsor,  and,  in  addition  RAND  constantly 
examines  the  corporate  overhead  rates. 

The  Review  concluded  that  the  Center  met  all  the  needs  of  the  Air  Force,  and  recommended  renewal. 


Box  E-1 :  Project  AIR  FORCE  Assessment 

Project  Name: 

RPN:  FY  93  MTS:  Project  Status: 

Sponsor:  Project  Officer: 

AWARD  FEE  EVALUATION  AREAS  AND  CRITERIA 
AREA  1 :  COMMUNICATION 

Assesses  RAND/PAF’s  interaction  with  the  customer.  Accounts  for  60%  of  the  total  score.  This  category  considers, 
but  is  not  limited  to: 

•  Initiative  and  Cooperation:  RAND/PAF’s  ability  to  interrelate  with  the  Air  Force  to  develop  a  rapport  that  results 
in  mutually  agreeable  methods  of  attaining  research  objectives. 

•  Responsiveness:  RAND/PAF’s  ability  to  respond  to  sponsor  queries  and  concerns,  with  special  emphasis  on 
quality  and  rapidity. 

•  Timeliness  and  Frequency:  RAND/PAF’s  ability  to  keep  the  sponsor  apprised  of  the  status,  direction,  and 
issues  concerning  on-going  research,  and  the  ability  to  provide  a  meaningful,  two-way  forum. 

AREA  2:  RESEARCH  CLARITY  &  UTILITY 

Assesses  the  quality  of  RAND/PAF’s  research,  accounts  for  25%  of  the  score.  This  category  considers,  but  is  not 
limited  to: 

•  Clarity  of  result:  Data  and  analysis  support  the  conclusions  of  the  research. 

•  Utility:  The  research  is  pertinent  to  Air  Force  policy  making  and/or  decisionmaking. 

AREA  3:  EFFICIENCY/EFFECTIVENESS 

Assesses  RAND/PAF’s  management  effectiveness.  Accounts  for  15%  of  the  total  score.  It  considers,  but  is  not 
limited  to: 

•  Cost  control:  the  contractor’s  ability  to  control  costs. 

•  Data:  the  contractor’s  ability  to  submit  data  in  a  timely  manner  as  requested  by  the  government. 
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THE  ARROYO  CENTER 


Objectives  and  guidelines  for  scope  of  work ,  organizational  structure,  size  and  appropriateness  of 
customers . 

The  Arroyo  Center  was  transferred  from  the  Jet  Propulsion  Laboratory  to  RAND  in  1 984.  It  is  the 
Army’s  only  studies  and  analyses  FFRDC.  The  Center  has  a  MTS  of  104  professionals.  The  stated 
mission  of  the  Arroyo  Center  is  to  provide  objective,  independent  analytic  research  on  major  policy, 
technical,  and  management  issues  that  represent  mid-  to  long-term  concerns  for  the  Army.  The  Arroyo 
Center  also  provides  a  balanced,  arms-length  perspective  on  current  controversial  problems  while 
maintaining  expertise  on  and  institutional  memory  about  Army  operations  and  policy. 

The  organizational  structure  is  shown  in  figure  E-9. 


Figure  E-9:  Arroyo  FFRDC 


SOURCE:  RAND 

The  bulk  of  the  Arroyo's  funding  comes  from  a  single  program  element  in  the  Army  s  defense  budget. 
In  FY  1995,  the  Arroyo  Center  had  a  funding  level  of  $20.1  million.  Funding  has  declined  steadily 
from  its  peak  of  $25.5  million  in  1990  (figure  E-10). 

The  Center  receives  funding  through  two  channels,  a  CORE  Program  Element  and  customer 
reimbursable  funds.  The  CORE  funding  is  based  on  a  specified  level  of  effort.  This  funding  is  entered 
into  the  programming  and  budgeting  cycle  and  is  projected  for  six  years.  The  second  method  of 
receiving  funds  is  through  customer-reimbursable  funds.  In  this  case  a  customer  requests  support  and 
transfers  funds  to  pay  for  the  research.  These  efforts  must  meet  the  criteria  established  for  the  use  of 
the  Arroyo  Center,  be  free  of  conflict  of  interest,  and  be  approved  by  the  Executive  Agent.  The  total 
annual  funding  is  now  limited  by  the  congressionaUy-imposed  ceiling. 

Funding  for  the  Arroyo  Center  is  not  limited  to  RDT&E  funds.  It  may  cross  several  program  elements 
or  appropriations. 
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Figure  E-10:  Arroyo  Funding  Profile  Actual  Obligations 


FY 


SOURCE:  Department  of  Defense 

Core  work  is  divided  into  four  programs  corresponding  to  the  Center’s  four  areas  of  core 
competence:  1)  Strategy  and  Doctrine,  2)  Force  Development  and  Technology,  3)  Military 
Logistics,  and  4)  Manpower  and  Training.  These  Core  Competencies  are  briefly  outlined  below. 

•  Force  Development  and  Technology.  The  Arroyo  Center  identifies  and  assesses 
ways  in  which  technological  advances  and  new  operational  concepts  can  improve  the 
Army’s  ability  to  meet  a  range  of  operational  demands  in  the  future.  This  work 
involves  assessments  of  a  technology’s  feasibility,  performance,  cost,  and  risk.  It 
focuses  on  helping  the  Army  determine  how  to  maintain  its  technological  edge. 

•  Manpower  and  Training.  Maintaining  high-quality  soldiers  requires  personnel 
policies  that  accomplish  the  key  objective  of  the  military  personnel  system:  attracting 
and  maintaining  the  right  people,  and  then  training  and  managing  them  in  a  way  that 
maximizes  their  capabilities. 

•  Military  Logistics.  Under  this  core  competency,  the  Arroyo  Center  identifies  and 
analyzes  researchable  issues  with  a  view  toward  how  improvements  in  the  Army’s 
logistics  capabilities  can  help  maximize  its  combat  effectiveness,  given  current  and 
likely  future  constraints  on  resources.  Work  in  this  area  addresses  the  problems 
involved  in  supporting  and  providing  resources  for  Army  combat  units,  the  logistics 
system,  and  the  sustaining  base. 

•  Strategy  and  Doctrine.  The  Arroyo  Center  provides  analyses  of  problems  in  the 
international  environment  that  affect  U.S.  national  security  in  general  and  Army 
operations  in  particular.  To  this  end,  Arroyo  conducts  research  on  emerging  threats 
to  U.S.  interests  and  focuses,  in  particular,  on  what  the  dramatically  shifting  power 
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relationships  in  the  post-Cold  War  world  will  mean  for  future  strategic  concepts,  for 
security  planning,  and  for  the  Army’s  future  roles,  structure,  and  doctrine.  In  terms 
of  core  competence,  the  Arroyo  Center  and  RAND  maintain  regional  expertise  to 
provide  threat  assessments  involving  long-term  political  and  military  developments  in 
regions  where  U.S.  national  security  interests  are  or  may  be  involved. 

The  Army  and  the  Arroyo  Center  state  that  the  Center  only  does  core  work.  Indeed,  only  core  work 
can  emerge  from  the  Arroyo  oversight  process.  The  Center  reports  that: 

For  any  given  fiscal  year,  what  constitutes  that  year’s  core  work  in  the  Arroyo 
Center  is  determined  in  advance  by  the  senior  leadership  of  the  Army  according 
to  a  specific  set  of  procedures.  Core  work  is  defined  annually  in  the  process  of 
these  leaders  applying  special  “strategic  relationship”  criteria  and  other  work- 
filtering  requirements  to  proposed  Arroyo  Center  projects,  as  discussed  below; 
deciding  which  projects  meet  the  requirements  and  which  of  those  can  be 
accomplished  within  available  resources;  and,  finally,  approving  the  overall  list 
of  Arroyo  projects  for  the  forthcoming  year.” 

The  criteria  for  determining  core  work  is: 

•  Consistency  with  Arroyo’s  Mission,  Purpose,  and  Capabilities 

•  Consistency  with  Arroyo’s  Core  Competencies 

•  Consistency  with  Arroyo’s  Special  FFRDC  Relationship 

The  level  and  appropriateness  of  non-DoD  work,  and  non-sponsored  work  undertaken  by  the 
FFRDCs  and  UARCs. 

Government  work  for  other  than  the  Army  must  be  approved  by  the  Arroyo  Center  Policy  Committee. 
There  is  currently  no  non-DoD  work  conducted  at  the  Arroyo  Center.  The  sponsoring  agreement 
stipulates  that  the  Center  “will  perform  no  work  for  private  industry  or  foreign  governments.” 

Compliance  with  the  DoD  Management  Plan 

The  DoD  Management  Plan  states  that  annual  levels  of  effort  shall  be  based  on  application  of  the 
core  concept  and  for  S&A  FFRDCs  guidelines  that  include:  (1)  maintaining  a  relatively  stable 
level-of-effort,  and  (2)  maintaining  competency  in  core  areas.  Staff  years  of  technical  effort 
(STEs)  are  used  in  sizing  and  managing  DoD-funded  FFRDC  work. 

The  Plan  states  that  DoD  FFRDCs  may  only  perform  core  work,  and  that  such  work  must  be 
approved  by  the  primary  sponsor;  can  only  be  accomplished  for  DoD,  other  government  entities 
(including  state  and  municipal  governments),  and  for  not-for-profit  activities.  It  cannot  include 
commercial  work. 

Parent  institutions  operating  DoD  FFRDCs  may  perform  non-FRDC  work,  but  such  work  must 
conform  to  some  specific  guidelines:  be  in  the  national  interest;  not  undermine  the  independence. 
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objectivity,  and  credibility  of  the  institute  or  call  into  question  the  freedom  from  conflict  of 
interest;  not  take  unfair  advantage  of  FFRDC  work;  and  normally  work  in  the  public  sector, 
although  commercial  work  may  be  accepted  if  the  sponsor  agrees. 

Fees  may  be  authorized.  The  Plan  calls  for  “an  annual  fee  proposal  justifying  each  element  of 
fee.” 

The  Army  and  the  Arroyo  Center  have  outlined  and  reported  four  areas  of  core  competency.  The 
Army  has  stated  in  its  latest  Arroyo  Center  acquisition  plan  that  104  MTS  will  satisfy  Army 
FFRDC  needs. 

Fees  are  authorized.  The  Army  has  anticipated  an  annual  fee  of  6%  in  its  projections,  but  RAND 
must  submit  a  need  for  fee  proposal.  RAND  reports  that  the  FY  96  fee  is  less  than  5.25%  and 
that  it  does  not  project  the  fee  to  be  as  high  as  6.0%  anytime  during  the  current  contract. 

The  level  of  funding  has  been  declining.  This  has  reportedly  had  an  impact  on  the  ability  to  retain  a 
critical  mass  in  some  areas. 

No  non-core  work  will  be  done  because  all  work  done  is  defined  as  core  work. 

Sponsor’s  Management  Processes 

Oversight  is  provided  by  an  Arroyo  Center  Policy  committee.  The  Committee  consists  of  a  mix  of 
personnel  from  Department  of  Army  organizations  and  U.S.  Army  commands  (table  E— 4).  The 
committee  is  co-chaired  by  the  Vice  Chief  of  Staff  of  the  Army  and  the  Assistant  Secretary  of  the  Army 
for  Research,  Development  and  Acquisition.  Most  of  the  personnel  on  the  board  are  general  officers. 


Table  E-4:  Arroyo  Center  Policy  Committee 

Vice  Chief  of  Staff  of  the  Army 
Deputy  Chief  of  Staff  Operations 
Deputy  Chief  of  Staff  Personnel 
Deputy  Chief  of  Staff  Logistics 

Director  Program  Analysis  and  Evaluation  (Executive  Agent) 

Assistant  Secretary  of  the  Army  (Research  Development  and  Acquisition) 
Assistant  Secretary  of  the  Army  (MRA) 

DUSA  (OR) 

Deputy  Chief  of  Staff  Intelligence 
Commander  Forces  Command 
Commander  Army  Materiel  Command 
Commander  Training  and  Doctrine  Command 

Commander  USA  Special  Operation  Command _ 


SOURCE:  U.S.  Army 

The  Army  leadership  determines,  in  advance  of  the  next  fiscal  year  and  prior  to  approving  an 
Arroyo  Center  work  program,  which  major  issues  the  Arroyo  Center  should  address.  This 
decision  is  made  by  the  .Arroyo  Center  Policy  Committee. 
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Once  the  ACPC  has  indicated  which  major  issues  the  Arroyo  Center  should  focus  on  during  the 
forthcoming  fiscal  year,  the  Arroyo  Center  develops  a  research  agenda  of  potential  studies  and 
analyses  that  is  rooted  in  the  ACPC’s  guidance.  This  agenda  is  then  compared  with  the  work 
program  developed  by  the  Army’s  Studies  Program  Coordination  Committee  (SPCC),  which  is 
chaired  by  the  Deputy  Undersecretary  of  the  Army  for  Operations  Research,  who  also  sits  on  the 
ACPC.  The  SPCC  oversees  Army  studies  and  analyses  undertaken  both  by  in-house  Army  assets 
and  by  external  (e.g.,  for-profit)  sources.  The  purpose  of  comparing  the  Arroyo  Center-proposed 
and  SPCC-monitored  work  is  to  eliminate  unnecessary  duplication  of  research  efforts  and 
resources,  to  re-filter  the  assignment  of  core  and  non-core  work  within  the  Army,  and  to  review 
the  application  of  the  criteria  for  assigning  core  work  to  an  FFRDC.  Following  this  comparison 
and  review,  the  ACPC  meets  to  approve  or  modify  the  Arroyo  Center’s  proposed  research 
agenda. 

The  Army  conducts  an  annual  project  survey  at  the  conclusion  of  each  research  cycle.  The  survey 
asks  each  project  sponsor  to  evaluate  the  objectives  of  the  research,  timeliness  of  the  products 
produced,  overall  quality  of  the  research  effort,  credibility  of  the  results,  methods  for  analysis, 
ability  to  display  data  in  a  meaningful  manner,  and  usefulness  of  the  products.  The  survey  also 
asks  sponsors  to  provide  an  overall  evaluation  of  the  Arroyo  Center.  These  surveys  are 
distributed  in  the  second  quarter  of  the  fiscal  year  for  evaluation  of  the  projects  completed  in  the 
previous  fiscal  year. 

According  to  the  Army,  the  Annual  Project  Survey  is  the  most  important  assessment  tool  for  the 
Army  in  evaluating  Arroyo  Center  performance.  The  surveys  provide  each  project  sponsor  with 
the  opportunity  to  evaluate  the  Arroyo  Center’s  performance  on  individual  project.  The  results  of 
these  surveys  are  briefed  to  the  Arroyo  Center  Policy  Committee  before  being  transmitted  to  the 
Arroyo  Center  Director  for  further  review. 

The  Army  reports  its  assessment  of  Arroyo’s  management  is  continuous  and  reviewed  twice 
annually,  spring  and  fall,  by  the  Arroyo  Center  Policy  Committee.  The  overall  operation  is 
governed  by  Army  Regulation  5-21. 

The  Army’s  assessment  to  ensure  a  cost-effective  operation  involves  activities  by: 

•  Procuring  Contract  Officer  (PCO)  assigned  to  Defense  Supply  Services  -  Washington 

•  Administrative  Contracting  Officer  (ACO)  located  in  El  Segundo,  CA 

•  Contracting  Officers  Representative  (COR)  assigned  to  the  Pentagon 

•  Defense  Contract  Audit  Agency  (DCAA)  located  in  Santa  Monica,  CA 

These  people  provide  review  and  oversight  over  the  day-to-day  operations  of  the  contract  and  the 
Center’s  management  controls. 
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Thoroughness  of  the  Five-Year  Reviews 

The  latest  Comprehensive  Review  of  the  Arroyo  Center  was  completed  in  December  1994.  The 
Review  examines  mission  needs,  alternative  sources,  effectiveness,  management,  and  costs.  The 
Army  states  that  it  was  conducted  in  accordance  with  OFPP  Policy  Letter  84-1  and  Federal 
Acquisition  Regulation  35.017. 

The  Review  outlines  current  Army  needs  by  topic  area  (Roles  &  Missions,  Force  Structure,  etc.) 
and  by  organization  (Assistant  Secretary  of  the  Army  for  Research,  Development  and  Acquisition, 
Forces  Command,  etc.)  The  document  notes  the  types  of  research  support  that  might  be  needed. 

It  matches  these  areas  to  the  broad  core  competency  areas  that  the  Army  has  asked  the  Center  to 
concentrate  upon.  It  concludes  that  the  Arroyo  Center  is  still  needed — at  a  level  of  104  MTS. 

The  Review  considers  in-house,  for-profit,  not-for-profit,  universities,  and  other  FFRDCs  as 
alternatives.  In-house  capabilities  are  dismissed  because  of  lower  salaries  for  equal  skills,  the 
influence  of  doctrine  and  training  on  research,  and  the  constant  turnover  that  would  accompany 
the  use  of  uniformed  personnel. 

The  Review  notes  that  for-profit  firms  already  “provide  over  50%  of  the  Army’s  study  and 
analysis  research.”  They  are  dismissed  in  this  case  because  they  may  have  limited  independence 
and  objectivity.  They  may  not  offer  the  interdisciplinary  approach,  long-term  perspective  and  in- 
depth  knowledge  that  Arroyo  now  has.  They  are  also  lacking  in  qualities  that  might  make  them  a 
“trusted  agent,”  and  are  not  given  “access  to  proprietary  and  otherwise  distribution-limited 
information.” 

Universities  are  dismissed  because  they  do  not  retain  the  full-time  dedicated  staff  with  the  range 
of  capabilities.  The  Review  notes  that  other  FFRDCs  now  provide  about  20%  of  the  research 
support  required  by  the  Army — but  these  are  primarily  in  special  areas  such  as  C3I.  The  Arroyo 
Center  has  all  the  attributes  the  Army  needs  and  desires. 

Effectiveness  and  efficiency  are  discussed  first  through  examples  of  research  on  different  topics, 
and  the  successful  use  of  that  research.  Efficiency  is  considered  in  the  context  of  the  results  of  the 
Army’s  Annual  Project  Survey  where  project  sponsors  are  asked  to  evaluate:  the  objectives  of 
the  research,  timeliness  of  the  products  produced,  overall  quality  of  the  research  effort,  credibility 
of  the  results,  methods  of  analysis,  ability  to  display  data  in  a  meaningful  manner,  and  usefulness 
of  the  products.  A  copy  of  the  survey  and  the  FY  91-93  results  are  shown  in  box  E-2.  The 
favorable  responses  show  a  decided  improvement  over  the  three  years  (figure  E-l  1). 
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Box  E-2:  Arroyo  Center  Project  Evaluation  Survey 


Project  Name:  ‘project* 


Sponsor:  ^address* 

Please  answer  the  following  questions  for  the  above  mentioned  project.  If  additional  paper  is  used  to  respond  to 
the  narrative  questions,  indicate  the  questions  being  addressed. 

Please  return  the  completed  survey  to:  Manpower  and  Force  Program  Analysis  Division 

ATTN:  MAJ  Wildasin  RM  3C179 
Chief  of  Staff 
200  Army  Pentagon 
Washington,  DC  20310-0200 


The  Arroyo  Center  developed 

objective  useful  to  the  sponsor  by: 

1.  Showing  an  understanding  of 
sponsor  problem. 

2.  Considering  alternative  views  of  all 
significant  groups  involved  in 
sponsor  process. 

3.  Developing  a  clear  statement  of 
purpose. 

4.  Investigating  the  background 
thoroughly. 

5.  Considering  all  explicit  and  implicit 
constraints  when  formulating  the 
problem  statement. 

6.  Outlining  the  problem  in  a  logical 
framework. 

7.  Defining  the  key  issues. 

8.  Creating  a  new  and  original 
perspective. 

The  Arroyo  Center  used  time 
efficiently  by: 

9.  Establishing  milestones  linked  to 
sponsor  requirements. 

10.  Completing  milestones  in  time  to 
make  a  difference. 


Strongly  Strongly  Unable  to 

Agree  Agree  Neutral  Disagree  Disagree  Evaluate 

A  B  C  D  E  X 


Strongly  Strongly  Unable  to 

Agree  Agree  Neutral  Disagree  Disagree  Evaluate 

A  B  C  D  E  X 
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The  Arroyo  Center  produced  a  quality  Strongly 

product  by:  Agree 

1 1 .  Developing  a  set  of  logical 
assumptions 

12.  Linking  assumptions  to  existing 
constraints. 

13.  Developing  an  adequate  range  of 
alternative  scenarios  and 
approaches  relative  to  sponsor's 
needs 

14.  Collecting  appropriate  information 
to  solve  the  problem. 

15.  Collecting  enough  information  to 
resolve  all  the  issues. 

16.  Using  appropriate  methods  to 
collect  information. 

17.  Collecting  data  without  disrupting 
sponsor/Army  operations. 

The  Arroyo  Center  maintained  Strongly 

credibility  by:  Agree 

18.  Considering  all  competing  views. 

19.  Applying  assumptions,  constraints, 
and  methods  consistently. 

20.  Defining  the  correct  criteria. 

21 .  Applying  a  suitable  measure  of  the 
criteria 

22.  Implementing  the  correct 
methodology. 

23.  Using  data  to  verify  the 
conclusions 

24.  Conducting  a  comprehensive 
literature  review. 

25.  Collecting  the  right  information. 

26.  Gathering  data  from  the  right 
sample. 

27.  Relating  this  data  to  similar  past 
work. 

28.  Coordinating  with  others  doing 
similar  work. 

29.  Avoiding  duplication. 


Agree 

B 


Agree 

B 


Strongly 

Neutral  Disagree  Disagree 

C  D  E 


Strongly 

Neutral  Disagree  Disagree 

C  D  E 


Unable  to 
Evaluate 
X 


Unable  to 
Evaluate 
X 
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The  Arroyo  Center  analyzed  the  data 


30.  Reducing  the  data  to  a 
comprehensive  level. 

31 .  Using  the  correct  analytical 
procedures. 

32.  Identifying  all  sources  of  data. 

33.  Including  all  pertinent  information. 

34.  Providing  a  statement  of  the 
reliability  of  the  data. 

35.  Furnishing  evidence  of  the  validity 
of  the  proposed  findings. 

The  Arroyo  Center  displayed  the  data 

by: 

36.  Presenting  findings  clearly  without 
ambiguity. 

37.  Balancing  the  use  of  text  and 
visual  aids. 


Strongly 

Agree 

A 


Strongly 

Agree 

A 


Agree 

B 


Agree 

B 


Strongly  Unable  to 
Neutral  Disagree  Disagree  Evaluate 

C  D  E  X 


Strongly  Unable  to 
Neutral  Disagree  Disagree  Evaluate 

C  D  E  X 


38.  Avoiding  the  use  of  jargon. 

39.  Ensuring  that  users  understood 
how  the  problem  statement, 
literature  review  objective, 
methods,  and  analyses  were  linked 
together. 

Use  of  Findings 

Yes  No  N/A 

40.  Did  the  Arroyo  Center  personnel  ensure  that  the  sponsor  knew  how  to  use 
the  findings/results  in  an  appropriate  manner? 

41 .  Did  you  as  the  sponsor  use  the  product  provided? 

42.  If  you  have  not  yet  used  the  product,  do  you  intend  to  ever  use  them? 

If  Yes,  please  explain  how  and  when. 

If  No,  please  explain  why  you  do  not  intend  to  use  product. 

Yes  No  N/A 

43.  In  your  opinion  were  the  findings  worth  the  level  of  the  effort? 

If  not,  why  not? 

44.  Would  you  use  the  contractor  again  in  the  same  manner? 

Please  Explain 

45.  What  would  you  do  differently  if  you  used  this  contrctor  in  the  future? 

46.  I  would  rate  the  Arroyo  Center  product  as: 

A.  Excellent 

B.  Above  Average 

C.  Average 

D.  Below  Average 
E  Poor 
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Figure  E-ll:  Arroyo  Survey  Results  %  of  Favorable  Response 

Met  Objectives 
Timliness  of  Results 

Quality  of  Product 

Credibility  of 
Research 

Data  Analysis 
Data  Display 

Usefulness  of  Results 

Overall  Research 
Quality 

0  20  40  60  80  100 


SOURCE:  U.S.  Army 

The  Review  also  assessed  the  Center’s  management  controls  to  ensure  a  cost-effective  operation. 
It  outlined  the  chains  of  oversight,  discussed  accounting  and  purchasing  systems  and  overhead 
costs  and  management  fees.  The  Review  noted  RAND  management  improvements  (principally 
reduction  in  travel  costs)  and  Army  oversight  actions  (steps  to  focus  research,  establishment  of  a 
working  group  to  help  set  the  research  agenda,  an  improved  final  review  and  coordination  of 
research)  aimed  at  cost  control. 

The  Review  concluded  that  the  Center  meets  all  the  needs  of  the  Army  and  recommended 
contract  renewal. 
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The  National  Defense  Research  Institute 

Objectives  and  guidelines  for  scope  of  work,  organizational  structure,  size,  and 
appropriateness  of  customers 

The  National  Defense  Research  Institute  (NDRI)  was  established  at  RAND  in  1984  as  a  separate 
FFRDC  to  consolidate  all  the  work  being  done  for  DoD  (other  than  that  being  done  for  the  Army 
and  Air  Force)  into  one  contract,  planning  process,  and  oversight  arrangement. 

The  FFRDC  is  currently  staffed  at  a  level  of  104  MTS,  along  with  appropriate  support  personnel.  For 
FY  1996,  a  NDRI  MTS  is  calculated  to  cost  $210,5773  The  most  recent  report  to  Congress  on  DoD 
FFRDC  management  forecasts  a  future  NDRI  staff  effort  of  106  STE  (Staff  Year  of  Technical 
Effort— the  new  term  for  Man-Year  of  Technical  Staff).  The  NDRI  was  recently  reorganized.  The 
new  organizational  structure  is  shown  in  figure  E-12. 


Figure  E-12:  National  Defense  Research  Institute  FFRDC 


SOURCE:  RAND 

NDRI  serves  the  long-term  analytic,  planning  and  innovation  needs  of  OSD,  the  Joint  Staff,  and 
other  defense  agencies.  It  is  intended  to  fulfill  the  following  objectives: 

•  to  conduct  a  sustained  research  program, 

•  to  establish  a  reservoir  of  expertise, 

•  to  allow  flexible  response,  and 

•  to  provide  unbiased  analysis. 

There  is  no  specific  budget  line  for  NDRI.  Instead,  users  fund  projects  as  the  projects  are  needed.  The 
primary  source  of  funds  is  the  OSD  studies  fund.  Alternate  sources  can  also  be  used.  While  the  Office 
of  the  Undersecretary  of  Defense  provides  most  of  the  funding,  over  20  different  DoD  organizations 
sponsor-studies  at  NDRI.  NDRI  funding  has  declined  noticeably  since  its  1990  peak,  and  in  real  terms 
current  funding  level  is  back  near  what  it  was  in  FY  1987  (figure  E-13). 


3  Calculated  with  a  $21.9  million  budget  for  the  104  MTS. 
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Figure  E-13:  NDRI  Funding  Profile 


NDRI’s  core  competencies  are  listed  in  table  E-5. 
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Table  E-5:  NDRI  "Core”  Competencies 


*  International  Strategy  and  Defense  Policy: 

•  Regional  expertise 

•  International  security  structures 

•  Defense  doctrine  and  strategy 

*  Forces  and  resources  policy: 

•  Force  planning  and  structure 

•  Personnel  management 

•  Training 

•  Health  care 

*  Acquisition  and  technology  policy: 

•  Weapons 

•  Information 

•  Other  critical  technologies 

•  Industrial  base  issues 


•  Non-military  instruments  of  power 

•  Strategic  planning 

•  Budget  analysis 

•  Resource  management  and  resource  management 
models 

•  High-level  logistics 

•  Acquisition  policy 

•  Cost  Analysis 

•  Modeling  and  simulation  theory 


Source:  Office  of  the  Secretary  of  Defense _ _ _ _ _ 

The  NDRI  management  states  that  the  Center  does  not  do  non-core  work.  This  is  assured  through  a 
process  that  includes  review  by  the  NDRI  Advisory  Board.  The  process  is  discussed  later  under 
Sponsor’s  Management  Process.  The  process  for  accepting  work  is  outlined  in  table  E-6. 


Table  E-6:  NDRI  Work  Acceptance  Process 

* 

NDRI  Mission  and  Capabilities 
Core  Competencies 

—  International  strategy  and  defense  policy 

—  Forces  and  resources  policy 

—  Acquisition  and  technology  policy 

Strategic  Relationship 

_  Long-term  continuity  —  Familiarity  with  sponsor  missions 

—  Flexibility  and  adaptability  —  Responsiveness 

_  Independent  and  objective  —  Proprietary  and  sensitive  data 

Important  DoD  need 

Source:  Office  of  the  Secretary  of  Defense  _ _ _ 
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Work  must  be  appropriate  to  NDRI’s  mission  and  capabilities.  It  must  be  among  the  three  broad  areas 
of  core  competence.  It  must  be  in  accordance  with  the  strategic  relationship  with  their  clients,  and  the 
work  must  be  deemed  important  to  the  DoD.  The  criteria  for  project  selection  is  listed  below. 

(1)  Consistency  with  NDRI’s  mission,  purpose,  and  general  scope  of  effort; 

(2)  Applicability  of  special  expertise  or  competence  at  NDRI  that  would  indicate  it  is 
the  most  effective  source; 

(3)  Need  for  objectivity  and  independence,  and  freedom  from  real  or  perceived 
conflict  s-of-interest, 

(4)  Importance  and  complexity  of  the  problem  or  issue  in  the  sense  that  it  calls  for  an 
integrative,  cross-cutting,  or  interdisciplinary  approach; 

(5)  Relationship  to  and  support  for  the  goals  of  the  overall  NDRI  research  agenda, 
and  opportunity  for  complementary  with  other  NDRI  research  underway; 

(6)  Opportunity  for  innovation  and  research  capital  building  to  support  current  and 
future  needs  (including  quick-response  capability)  of  the  sponsor  community; 

(7)  Requirement  to  acquire  or  have  access  to  privileged,  proprietary,  competition- 
sensitive,  or  otherwise  restricted  data  (including  information  not  available  to 
contractors). 

The  level  and  appropriateness  of  non-DoD  work  and  work  for  other  than  the  primary  sponsor. 

NDRI  may  accept  work  for  non-DoD  federal  agencies  if  work  (1)  is  determined  to  be  consistent  with 
its  mission  and  within  its  general  scope,  (2)  is  in  compliance  with  FAR  Part  17.502  and  1 7.504(e),  and 
(3)  is  specifically  authorized  by  the  primary  sponsor.  Work  that  is  considered  to  be  non-core  work  will 
not  be  performed  in  the  NDRI  or  in  the  other  two  RAND  DoD  FFRDCs. 

Compliance  with  the  DoD  Management  Plan 

The  DoD  Management  Plan  states  that  annual  levels  of  effort  shall  be  based  on  application  of  core 
concept  and  for  the  S&A  FFRDCs  guidelines  that  include:  (1)  maintaining  a  relatively  stable  level- 
of-effort,  and  (2)  maintaining  competency  in  core  areas.  Staff  years  of  technical  effort  (STEs)  are 
used  in  sizing  and  managing  DoD-funded  FFRDC  work. 

The  Plan  states  that  DoD  FFRDCs  may  only  perform  core  work,  that  such  work  must  be 
approved  by  the  primary  sponsor;  can  only  be  accomplished  for  DoD,  other  government  entities 
(including  state  and  municipal  governments),  and  for  not-for-profit  activities.  It  cannot  include 
commercial  work. 


E-28 


Parent  institutions  operating  DoD  FFRDCs  may  perform  non-FFRDC  work,  but  such  work  must 
conform  to  some  specific  guidelines:  be  in  the  national  interest;  not  undermine  the  independence, 
objectivity,  and  credibility  of  the  institute  or  call  into  question  the  freedom  from  conflict  of 
interest;  not  take  unfair  advantage  of  FFRDC  work;  and  normally  work  in  the  public  sector, 
although  commercial  work  may  be  accepted  if  the  sponsor  agrees. 

Fees  may  be  authorized.  The  Plan  calls  for  “an  annual  fee  proposal  justifying  each  element  of 
fee.” 

The  sponsor  reports  that  the  NDRI  Advisory  Board  “ensures  funding  is  sufficient  to  retain  required 
core  capabilities.”  There  appears  to  be  some  difference  of  opinion  about  that  level.  The  level  of  effort 
has  been  relatively  stable,  but  a  RAND  document  reports  that: 

The  FY  1996-2000  research  effort  suggests  that  NDRI  requires  no  less  than  120  MTS  to 
maintain  essential  core  capabilities  and  carry  out  both  long-range  and  quick-response 
research  in  the  policy  issue  areas  specified  At  the  same  time,  120  MTS  provides  for  no 
significant  surge  capacity  in  any  area  without  curtailing  work  in  other  areas.  Experience 
shows  that  every  year  NDRI  is  asked  to  undertake  at  least  one  extraordinary  effort, 
whether  commissioned  by  the  Secretary  of  Defense  or  mandated  by  Congress.  DoD  and 
NDRI  management  can  accommodate  this  type  of  surge  or  special  need  by  committing 
funding  above  the  level  required  to  maintain  120  MTS.  However,  below  120  MTS,  not 
all  policy  issue  areas  can  be  supported  to  the  depth  and  degree  of  excellence  expected  of 
a  studies  and  analysis  FFRDC,  and  the  sponsor  community  and  NDRI  advisory  board 
will  need  to  consider  which  issue  areas  will  not  receive  NDRFs  analytical  support. 

As  noted  above,  NDRI  conducts  only  core  work.  Work  for  other  than  OSD,  the  Joint  Staff  or  the 
defense  agencies  must  be  within  the  core  and  must  be  approved  by  the  primary  sponsor.  RAND  may 
do  other  work — and  does.  As  with  the  other  RAND  Centers,  a  fee  is  authorized. 

The  current  ceiling  is  104  MTS.  The  new  DoD  management  plan  envisions  106  STE. 

Sponsors  Management  Process 

The  primary  sponsor  for  NDRI  is  the  Under  Secretary  of  Defense  (Acquisition  and  Technology),  who 
is  responsible  for  ensuring  that  NDRI: 

•  Is  used  for  intended  purposes 

•  Costs  of  services  are  reasonable 

•  Produces  high-quality  work  of  value  to  users 

The  day-to-day  management  and  oversight  is  delegated  to  the  Director,  Acquisition  Program 
Integration.  The  Director: 


4  NDRI  Research  Plan  for  1996-2000,  June  1995,  AR-5935-OSD,  RAND  Corporation,  p.  6. 
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•  Designates  membership  and  chairs  NDRI  Advisory  Board 

•  Provides  day-to-day  management  through  FFRDC  program  office  and  Contracting 
Officers’  Technical  Representative 

The  NDRI  Advisory  Board 

•  Recommends  multi-year  level  of  effort  in  terms  of  professional  person-years 

•  Provides  information  as  to  DoD  research  needs,  interests  and  priorities,  to  include 
issues  that  cut  across  and  bridge  mutual  interests 

•  Reviews  and  approves  long-term  research  plan  and  annual  research 
program — ensuring  that  research  is  in  compliance  with  “core”  definition 

•  Identifies  funding  for  the  program  from  the  OSD  studies  budget  and  other  available 
sources 

•  Evaluates  semi-annually  with  NDRI  management  the  current  research  effort  and 
makes  corrections  as  needed 

•  Reviews  annual  assessment  of  individual  research  projects  and  in  coordination  with 
NDRI,  takes  action  to  resolve  problems 

The  NDRI  Advisory  Board  is  shown  in  figure  E-14. 


Figure  E-14:  NDRI  Advisory  Board 


SOURCE:  Office  of  the  Secretary'  of  Defense 

The  NDRI  Advisory  Board  constitutes  the  final  component  in  defining  NDRI’s  core  research. 

This  Board,  which  is  responsible  for  NDRI  governance,  oversees  the  research  planning  process. 

In  particular,  the  NDRI  reports  that  it  helps  develop  and  approves  the  long-term  research  plan  and 
the  annual  research  program,  which  constitutes  NDRI’s  core  research  for  the  year.  With  respect 
to  the  research  planning  process,  it  has  three  primary  responsibilities.  (1)  It  is  responsible  for 
ensuring  that  what  constitutes  NDRI’s  core  research  is  indeed  aligned  with  the  current 
requirements  of  the  sponsor  community.  As  those  requirements  change  with  time,  so  will  the 
details  of  what  constitutes  the  core  research  program.  (2)  The  Board  is  responsible  for  ensuring 
that  NDRI  is  used  only  for  essential  core  work.  (3)  It  is  responsible  for  ensuring  that  the 
capability  required  to  accomplish  this  core  work  is  maintained. 
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The  broad  range  of  sponsors  for  individual  projects  operate  with  different  centers  within  the  NDRI. 
This  is  illustrated  in  figure  E-15. 


Figure  E-15:  National  Defense  Research  Institute 
Center  -Sponsor  Alignment 
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The  Executive  Agent  conducts  an  annual  assessment  to  evaluate  technical  quality,  responsiveness, 
value,  cost,  and  timeliness  as  is  specified  in  the  DoD  FFRDC  Management  Plan. 

For  the  annual  assessment,  a  survey  of  proj  ect  sponsors  is  conducted.  The  survey  gathers  data  on 
sponsors’  perceptions  of  the  technical  quality,  responsiveness,  value,  and  timeliness  of  NDRI  work. 
The  Executive  Agent  compiles  and  analyze  the  survey  inputs. 

The  Executive  Agent  reports  the  results  of  the  assessment  to  the  Advisory  Board  and  to  NDRI.  The 
Advisory  Board  reviews  the  assessment  with  NDRI,  provides  feedback,  and  resolves  any  real  or 
perceived  problems. 

NDRI  also  tracks  the  timeliness  of  all  key  deliverables  specified  in  the  project  descriptions,  and  reports 
regularly  to  the  Executive  Agent  any  significant  delays  along  with  planned  remedial  actions. 

Costs  are  evaluated  annually  by  the  Defense  Contract  Audit  Agency  as  required  by  the  NDRI  contract 

Changes  in  the  details  of  the  annual  assessment  procedures  can  be  made  by  agreement  between  NDRI 
and  the  Advisory  Board. 
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Thoroughness  of  the  Five-Year  Review 

The  latest  NDRI  Comprehensive  Review  is  dated  October  1, 1994.  The  Review  examined  mission 
needs,  alternative  sources,  effectiveness,  management,  and  costs.  The  Sponsor  states  that  it  was 
conducted  in  accordance  with  OFPP  Policy  Letter  84-1  and  Federal  Acquisition  Regulations  35.017. 

The  Review  is  relatively  short.  It  addresses  sponsor  requirements  in  terms  of  the  varied  needs  of  the 
different  offices  using  NDRI.  These  needs,  it  says,  tend  to  cluster  into:  (1)  international  security  and 
defense  strategy,  (2)  manpower  and  other  resources,  and  (3)  acquisition  and  technology  policy.  The 
discussion  outlines  global  trends  in  these  three  areas  and  discusses  the  need  for  an  organization  that  can 
address  such  subjects,  and  also  has  a  set  of  general  attributes  as  listed  below: 

•  a  sustained,  integrated,  and  comprehensive  research  effort 

•  the  ability  to  perform  cross-cutting  and  integrative  analysis 

•  the  ability  to  produce  high  quality,  useful  research 

•  a  long  term  perspective 

•  an  interdisciplinary  research  approach 

•  is  responsive  and  flexible 

•  a  deep  understanding  of  sponsor  needs 

•  the  ability  to  generate  new  ideas  and  bring  fresh  perspectives 

•  modeling  and  simulation  capability 

Alternative  sources  are  very  briefly  considered,  but  dismissed  because  “NDRI  is  specifically  capable 
and  specialized  to  address  issues  that  require  cross-cutting,  integrated,  interdisciplinary  solutions,  and 
issues  that  require  a  higher  degree  of  independence,  objectivity,  and  freedom  from  real  or  potential 
conflicts  of  interests  than  other  sources  can  provide”  NDRI  is  not  used  if  other  sources  can  provide 
the  work.  The  Review  also  briefly  notes  the  standard  limitation  on  alternative  sources  in  general. 

The  discussion  of  effectiveness  and  efficiency  is  based  on  a  survey  sampling  of  76  randomly  selected 
completed  projects  (of  the  260  completed  during  the  contract).  The  results  showed  that  sponsors  were 
generally  “extremely  pleased  with  the  efforts  of  NDRI  and  all  recommended  renewal  of  the  contract.” 
Studies  have  been  viewed  as  of  “high  quality.”  The  Review  notes  that  NDRI  has  a  broad  range  of 
technical  capabilities,  and  a  responsive,  independent  research  voice. 

There  were  also  some  reported  criticisms.  Timeliness  was  identified  as  a  problem.  NDRI  proposed 
ways  to  improve  on  its  deliveiy. 

Management  controls  were  discussed  in  terms  of  general  oversight  from  OUSD  (A&T),  the  NDRI 
Advisory  Board,  and  its  working  group.  The  5-member  Advisory  Board  replaces  a  13-member 
Defense  Advisory  Group  that  was  viewed  as  ineffective. 

The  report  noted  that  DCAA  reviewed  RAND’s  management  of  NDRI,  and  found  RAND’s 
operations  adequate. 
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RAND  itself  has  reduced  the  NDR1  senior  management  structure,  improved  responsiveness  with  a 
Washington  presence,  improved  the  project  planning  and  approval  process,  and  lowered  cost  per  MTS 
3%  between  FY  93  and  FY  94. 

The  overall  Review  conclusion  was  that  the  NDRJ  was  performing  well  and  the  contract  should  be 
renewed. 
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Appendix  F 


The  MITRE  Corporation 

The  information  that  MITRE  and  its  DoD  FFRDC’s  primary  and  secondary  sponsors  presented  to 
the  Task  Force  is  summarized  in  this  appendix.  The  sponsors  were  very  supportive  of  the  type  of 
work  MITRE  did  for  them,  the  continued  need  for  that  work,  and  the  quality  of  METRE’S 
products. 

OVERVIEW 

The  objectives  and  guidelines  for  scope  of  work,  organizational  structure,  size, 
appropriateness  of  customers. 

The  MITRE  Corporation  currently  operates  two  FFRDCs:  the  DoD’s  C3I FFRDC  and  the 
Federal  Aviation  Administration’s  Center  for  Advanced  Aviation  System  Development.  About 
85%  of  its  current  work  program  is  in  the  C'T  FFRDC.  Within  the  C3I  FFRDC,  more  than  90% 
of  the  work  is  for  the  Department  of  Defense,  and  the  remainder  is  for  other  elements  of  the 
intelligence  community.  About  half  of  the  CT  FFRDC  effort  is  for  the  U.S.  Air  Force.  Prior  to 
the  division  of  the  Corporation  on  January  29,  1996,  MITRE  also  worked  for  a  variety  of  Federal 
agencies  and  state,  local,  and  foreign  governments. 

The  MITRE  Corporation  was  established  in  1958  as  a  non-profit  institution.  It  was  formed  from 
a  division  of  Lincoln  Laboratory  and  had  the  job  of  implementing  and  integrating  the  Semi- 
Automatic  Ground  Environment  (SAGE)  System  that  had  been  developed  at  Lincoln  Laboratory. 
Through  the  following  decades,  the  MITRE-operated  C3I  research  facility  has  been  a  central  point 
of  C'T  for  the  DoD.  The  facility  was  sponsored  solely  by  the  Air  Force— although  it  had  additional 
users— until  1990.  At  that  time,  the  Assistant  Secretary  of  Defense  for  Command,  Control, 
Communications  and  Intelligence  (ASD(C3I))  was  designated  the  primary  sponsor  of  the  MITRE 
C3I  FFRDC,  with  the  Air  Force  designated  as  the  sponsor  of  the  MITRE  Bedford  C3I  Division 
and  the  Army  designated  as  sponsor  of  the  MITRE  Washington  C3I  Division.  Until  early  1996, 
MITRE’s  corporate  structure  consisted  of  two  FFRDCs  (one  DoD  and  one  FAA)  and  two  other 
operating  centers  (the  Center  for  Information  Systems,  and  the  Environment,  Resource,  and 
Space  Center).  In  1996, the  non-FFRDC  civil  work  was  split  off  to  form  Mitretek  Systems,  Inc. 

(a  non-profit  company),  leaving  MITRE  with  two  FFRDCs  (see  figure  F-l). 


Figure  F-l:  The  MITRE  Corporation 
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The  ASD(CT)  who  is  the  primary  sponsor,  the  Air  Force,  and  the  Army  work  together  as  a  team 
to  manage  the  Government’s  use  of  the  C3I FFRDC.  The  ASD(CT)  and  the  ASA  (RDA)  jointly 
chair  the  Joint  Users’  Committee  (JUC)  which  was  established  in  1990.  The  JUC  sets  priorities 
for  the  FFRDC  by  allocating  its  ceiling  among  various  DoD  elements.  The  Air  Force  recently 
began  participating  on  the  Committee.  The  Air  Force  also  retains  its  review  of  MITRE  in  the 
context  of  its  contract  with  the  Center.  The  Air  Force  (through  its  Electronic  Systems  Center 
(ESC))  and  the  Army  (through  its  Communications-Electronics  Command  (CECOM))  each 
manage  a  major  contract  with  the  MITRE  Corporation;  in  general.  Air  Force  work  is  funded 
through  the  ESC  contract  and  other  DoD  work  is  funded  through  the  CECOM  contract. 
Management  direction  and  oversight  is  provided  by  the  various  users,  who  fund  MITRE  activities 
from  their  own  budgets.  The  ASD(CsI),  ESC,  and  CECOM  work  together  to  provide  oversight 
and  quality  assurance  over  the  FFRDC ’s  work  (see  figure  F-2). 
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Figure  F-2:  Government  Management  of  MITRE 


SOURCE:  Department  of  Defense 

MITRE  is  intimately  integrated  with  the  Air  Force’s  ESC.  The  Center  shows  MITRE  as  one  of 
three  elements  of  its  work  force:  49%  are  in-house  military  and  civil  service  personnel,  27%  are 
other  support  contractors,  and  MITRE  is  24%;  MITRE  represents  44%  of  the  ESC  technical 
work  force.  ESC  stated  that  the  organization  does  not  feel  that  it  is  being  allowed  to  use  MITRE 
as  much  as  needed.  Similarly,  the  other  users  of  the  C3I FFRDC  have  work  demands  that  exceed 
the  ceiling. 

The  DoD  work  mix  for  the  C3I  FFRDC  is  shown  in  figure  F-3. 
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Figure  F-3:  C3!  FFRDC  Work  Distribution 
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The  sponsoring  agreement  conforms  to  the  FAR  requirement  that  the  total  non-U.S.  Government 
funding  (including  CRADAs)  not  exceed  30%  of  the  total  FFRDC’s  funded  support.  The  funding 
profile  for  MITRE  is  shown  in  figure  F-4. 


Figure  F-4:  MITRE  Funding  Profile  Actual  Obligations 
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Since  1993,  MITRE’s  total  revenues  have  declined.  MITRE’s  estimated  1996  revenues  after 
splitting  the  company  are  $455  million,  including  work  in  both  the  DoD  and  the  FAA  FFRDCs. 
The  DoD’s  FY  96  ceiling  allocation  is  $341.6  million.  The  FY  96  ceiling  allocations  for  the  C3I 
FFRDC  are: 

Air  Force  Contract  $1 34,323,000 

Army  Contract 


Army 

$47,984,700 

Navy 

36,233,800 

DISA 

27,245,900 

NS  A 
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Joint  Staff/CINC 

14,460,000 

DIA 

3,470,900 

CIO 

2,192,200 

OASD  (C3I) 

5,998,000 

Other  DoD 

7,281,500 

Total  Army  Contract  157,287,000 

$341,610,000 

In  addition,  the  C3I  FFRDC’s  FY1996  work  program  is  expected  to  include  non-ceiling  projects 
that  total  about  $50  million.  Customers  are  the  CIA  and  countries  engaged  with  DoD  in  foreign 
military  sales. 

MITRE  has  a  total  employment  of  about  4,700.  About  2,750  are  technical  staff.  A  total  of  2,300 
MTS  are  assigned  to  the  C3I  FFRDC.  Among  the  total  MITRE  technical  staff,  41  percent  are 
electrical  engineers  and  other  types  of  engineers,  21  percent  are  computer  scientists,  and  20 
percent  are  physicists  and  mathematicians  (see  figure  F-5). 


Figure  F-5:  Composition  of  MITRE  Research  Staff 
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SOURCE:  MITRE 

The  total  cost  of  all  MITRE  C31 FFRDC  work,  divided  by  the  number  of  technical  staff  years  of 
effort  delivered,  is  about  $189,000.  MITRE  notes  that  this  includes  a  variety  of  direct  costs  such 
as  project-related  travel,  equipment  purchases,  and  support  labor,  which  private  industry  does  not 
normally  include  when  it  quotes  a  “cost  per  staff  year.”  This  cost,  when  adjusted  for  inflation,  has 
declined  each  year  since  1991 .  ESC  noted  that  its  analysis  of  burdened  staff  costs  showed  that 
MITRE  is  comparable  to  commercial  contractors  for  individuals  with  equivalent  skill  and 
experience. 

The  C3I  FFRDC’s  mission  is  to  provide  the  DoD  Command,  Control,  Communications, 
Computers  and  Intelligence,  Surveillance  and  Reconnaissance  (C4ISR)  general  systems 
engineering  task,  engineering  support  and  systems  integration  support  for  the  Services,  the 
Unified  Commands^  the  Joint  Staff,  the  Defense  Agencies  and  the  Office  of  the  Secretary  of 
Defense.  The  purpose  of  the  C3I  FFRDC  is  to  provide  the  DoD  the  capability  to  access  the 
private  sector’s  systems  engineering  capabilities  from  an  organization  which  operates  in  the  public 
interest.  The  scope  of  the  C3I  FFRDC  activity  extends  across  the  functional  responsibilities  of  the 
C3I  mission  area  on  a  worldwide  basis  and  includes  systems  research  and  planning,  system  design, 
technical  management,  research  and  experimentation  in  selected  fields.  The  sponsoring  agreement 
states  that,  “While  serving  the  immediate  needs  of  the  many  individual  work  programs  it  supports, 
the  C31  FFRDC  directs  its  work  program  towards  achieving  DoD’s  vision  of  an  integrated  C4ISR 
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capability  based  upon  a  system  of  systems  that  supports  joint  U.S.  and  coalition  military 
operations.” 

MITRE  reports  that  an  extraordinary  depth  and  breadth  of  knowledge  and  experience  in  the  C  I 
domain  and  its  underlying  enabling  technologies,  together  with  expertise  in  operational  needs 
analysis,  engineering  processes  and  integration  techniques,  form  the  basis  of  MITRE  s  unique 
core  competencies.  The  C3I FFRDC  core  competencies  are: 

System  of  Systems  Engineering:  Provides  the  architecture  and  planning  and  development, 
internal  and  external  interface  definition,  modeling  and  simulation  analysis,  and  integration  and 
testing  necessary  to  support  the  development  of  a  single,  integrated  C  I  system  of  systems.  This 
single  integrated  system  provides  for  interoperability  across  all  systems  to  support  effective  joint 
operations. 

Systems  Development  and  Acquisition:  Provides  operational  requirements  analysis  and 
evaluation,  mission  threat  analysis,  risk  assessment,  and  technical  performance  analysis  and 
assessment  to  support  acquisition  planning,  program  preparation  and  evaluation,  and  program 
milestone  and  design  reviews  for  sill  C4I  systems.  Additionally,  it  provides  a  technical,  unbiased 
perspective,  not  available  in  the  for-profit  sector,  on  emerging  C  I  technologies,  available 
commercial  off-the-shelf  products,  and  associated  implementation  risks  with  the  development  and 
acquisition  of  complex  C4I  systems. 

Process  Implementation:  Provides  technical  expertise  to  support  acquisition  reform  initiatives 
such  as  military  specifications  and  standards  reform,  as  well  as  to  support  proof  of  concept 
prototyping  in  support  of  C4I  systems.  The  C3I  FFRDC  provides  the  broad  technical  expertise  on 
available  and  emerging  computer-based  tools  to  enable  realistic  definition  of  complex  systems 
architectures  required  to  improve  system  performance  and  implementation. 

Technology  Application:  Provides  state-of-the-art  assessments  of  opportunities,  alternatives, 
and  risks  to  support  technology  application  in  current  or  developing  C4I  systems. 

Additionally,  it  provides  the  expertise  and  broad  technical  background  to  assess  emerging 
technologies,  and  to  address  critical  C4I  problems  of  present  and  future  systems. 

Architectures  and  Interoperability:  Provides  an  in-depth  knowledge  of  emerging  Cl 
technologies  and  an  end-to-end  understanding  of  the  C41  mission  area  as  they  relate  to  C  I 
architectures  and  interoperability.  The  C3I  workforce  possesses  competencies  in  migration  of 
legacy  systems,  integration  of  operations  and  intelligence,  sustainment  and  modernization. 

Core  work  is  defined  as  that  which  meets  a  three-part  test.  Core  work  must  (1)  be  consistent 
with  the  C3I  FFRDC’ s  mission,  purpose,  scope,  and  capabilities;  (2)  be  within  the  C  I 
FFRDC’ s  core  competencies;  and  (3)  adhere  to  the  criteria  which  collectively  comprise  the 
special  relationship  between  the  DoD  and  the  C3I  FFRDC. 

The  criteria  for  selection  of  work  are  shown  in  figure  F-6. 
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Figure  F-6:  Criteria  for  Work  Assignment 


SOURCE:  Department  of  Defense 

The  Air  Force  selection  process  was  revised  and  core  functions  were  developed  in  1994.  All  work 
is  examined,  priorities  are  set,  impact  and  analysis  are  completed  and  a  “cuf’line  of  work  is 
established.  A  general  officer  or  civilian  equivalent  is  required  for  approval  to  initiate  new  work. 
Army  criteria  for  work  selection  are  similar. 

The  level  and  appropriateness  of  non-DoD  work,  and  non-Sponsored  work  undertaken  by  the 
FFRDCs  and  UARCs 

MITRE  can  accept  non-core  work,  but  only  if  approved  in  advance  by  the  primary  sponsor  and  if 
the  work  meets  the  acceptance  criteria  set  forth  in  the  sponsoring  agreement.  Only  core  work  is 
conducted  in  the  C3I FFRDC 

Any  C3I  FFRDC  work  performed  for  non-DoD  users,  including  foreign  governments,  requires 
prior  written  approval  of  the  primary  sponsor.  The  primary  sponsor  may  permit  the  work  through 
either  a  direct  purchase  or  a  transfer  of  funds  to  either  the  Air  Force  or  the  Army. 

Compliance  with  the  DoD  Management  Plan 

The  DoD  Management  Plan  guidelines  for  Systems  Engineering  and  Integration  Centers  include: 

•  Maintaining  stable  core  competencies 

•  Responding  to  projected  trends  in  workload  and  funding,  consistent  with  the  budget 
supporting  the  mission  area 
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The  Plan  states  that  DoD  FFRDCs  may  perform  only  core  work,  and  that  such  work  must  be 
approved  by  the  primary  sponsor;  can  be  accomplished  only  for  DoD,  other  government  entities 
(including  state  and  municipal  governments)  and  not-for-profit  activities;  and  cannot  include 
commercial  work. 

Parent  institutions  operating  DoD  FFRDCs  may  perform  non-FFRDC  work,  but  such  work  must 
conform  to  some  specific  guidelines:  be  in  the  national  interest;  not  undermine  the  independence, 
objectivity,  and  credibility  of  the  institution  or  call  into  question  the  freedom  from  conflict  on 
interest;  not  take  unfair  advantage  of  FFRDC  work;  and  normally  be  in  the  public  sector,  although 
commercial  work  may  be  accepted  if  the  sponsor  agrees. 

Fees  may  be  authorized.  The  Plan  calls  for  “an  annual  fee  proposal  justifying  the  need  for  each 
element  of  fee.” 

MITRE  reports  that  the  funding  ceiling  has  been  too  low  for  several  years,  a  problem  exacerbated 
by  repeated  Congressional  actions  to  reduce  the  ceiling  further.  There  is  a  “queue”  of  core  work 
for  which  funding  is  available  but  precluded  by  the  ceiling.  In  FY1996  this  queue  was  on  the 
order  of  one-third  of  the  size  of  the  ceiling.  Nevertheless,  if  Congress  permits  DoD  to  implement 
its  plan  to  maintain  a  relatively  stable  level  of  effort  over  the  next  few  years,  MITRE  believes  that 
it  can  maintain  its  core  competencies  and  provide  quality  work  for  those  DoD  programs  able  to 
justify  their  work  within  the  assigned  ceiling.  Apart  from  the  loss  of  value  caused  by  the  inability 
to  fit  all  core  work  under  the  ceiling,  MITRE  reports  that  the  reductions  in  ceiling  and  the 
accompanying  reductions  in  force  (RTFs)  have  increased  voluntary  attrition,  as  qualified  personnel 
have  concluded  that  their  career  prospects  would  be  brighter  elsewhere.  MITRE  is  concerned 
about  this  trend. 

The  C3I  FFRDC’s  focus  is  on  work  that  cannot  be  done  elsewhere.  As  figure  F-6  shows,  the 
work  done  at  MITRE  is,  by  definition,  done  only  at  MITRE.  For  example,  the  Air  Force  states 
that  it  will: 

Only  assign  tasks  to  MITRE  when  the  role  is  appropriate.  MITRE  will  not  be  used  if  the 
government  or  industry  can  do  the  job  effectively  and  without  a  conflict  of  interest.  The 
approved  Technical  Objectives  and  Plans  (TO&P)  must  clearly  define  and  document  a 
single  MITRE  project  role,  and  define  MITRE’s  relationships  with  other  contractors  on 
projects  or  programs  in  which  MITRE  has  a  role.  In  addition,  the  TO&P  must  be 
formalized  within  an  appropriate  enabling  clause  in  the  pertinent  contract. 

As  noted  above,  MITRE  can  perform  non-core,  non-FFRDC  work.  The  guidelines  in  the 
sponsoring  agreement  conform  to  those  stated  in  the  DoD  Management  Plan. 

Fees  are  authorized.  The  government  has  some  input  into  the  use  of  fees.  The  Air  Force  has 
quarterly  reviews  of  fees.  The  fees  are  negotiated  annually.  A  comparison  of  actual  and 
anticipated  fee  use  is  made  in  the  next  year’s  negotiation.  Base  fee  (ordinary  and  necessary 
expenses)  has  been  cut  to  less  than  1%. 
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According  to  the  primary  sponsor,  “the  sponsors  plan  to  maintain  a  long-term,  stable  C3I FFRDC, 
which  will  respond  to  changes  in  workload  and  funding  consistent  with  the  trend  in  research, 
development,  and  procurement  positions  of  the  DoD  budget  for  the  programs/systems  within  the 
C4ISR  mission  area.” 

Sponsor’s  Management  Process 

The  ASD  (C3I):  (1)  implements  policy  guidance  received  from  Congress,  SecDef,  USD(S&T), 
and  DDR&E;  (2)  provides  management  oversight  and  direction,  technical  review,  and  policy 
guidance  for  the  MITRE  C3I  FFRDC;  (3)  defines  the  MITRE  C3I  FFRDC  core  work  and  sets 
overall  priorities;  (4)  allocates  and  reallocates  dollar  ceiling  on  the  basis  of  inputs  from  DDR&E 
and  advice  from  the  Joint  Users  Committee  (JUC)  on  user  needs;  (5)  provides  guidance  to  the  Air 
Force  and  Army  (secondary  sponsors)  on  the  use  of  the  Cl  FFRDC  and  other  policy  matters 
requiring  enforcement  by  means  of  contract  provisions  or  actions;  (6)  enters  into,  interprets,  and 
modifies  as  necessary  the  FFRDC  sponsoring  agreement,  which  defines  the  mission  of  the  C3I 
FFRDC  and  the  mutual  obligations  arising  out  of  the  special  relationship  between  the  FFRDC,  the 
MITRE  Corporation,  and  DoD;  (7)  receives  input  from  the  JUC  and  from  various  auditors  on  the 
quality  of  the  Cl  FFRDC  work  and  management;  (8)  conducts  comprehensive  reviews  of  the 
MITRE  C3I  FFRDC  at  least  once  every  5  years;  and  (9)  serves  as  focal  point  for  FFRDC 
interactions  within  DoD. 

The  JUC  reviews  the  needs  for  the  MITRE  C3I  FFRDC  and  makes  input  to  ASD  (C3I)  on  ceiling 
allocation  requirements,  reviews  user  feedback  on  the  C3I  FFRDC’s  performance,  and 
recommends  areas  needing  attention  by  ASD  (C3I)  and/or  MITRE  management.  The  Research 
and  Technology  Subcommittee  reviews  proposed  C3I  FFRDC  “mission-oriented  investigation  and 
experimentation”  (MOIE)  projects  in  the  light  of  user  needs  and  determine  the  annual  MOIE 
program. 

The  Air  Force  Electronic  Systems  Center  (ESC)  and  Army  Communications  Electronics 
Command  (CECOM)  serve  as  secondary  sponsors.  They  (1)  negotiate  and  administer  contracts; 
(2)  manage  the  day-to-day  contracting  relationships  with  the  C3I  FFRDC;  (3)  act  as  focal  points 
to  assemble  and  prioritize  Air  Force  and  other  DoD  (respectively)  needs  for  C3I  FFRDC  support; 
(4)  review  all  government  requests  for  support  from  the  C3I  FFRDC  to  determine  whether  there  is 
adequate  justification  for  making  use  of  an  FFRDC;  (5)  review  documentation  for  C3I  FFRDC 
work  to  ensure  conformance  with  regulations  and  core  definition;  (6)  obtain  and  analyze  customer 
satisfaction  information  (“report  cards”)  and  provide  assessment  to  the  primary  and  secondary 
sponsors  as  well  as  to  MITRE  management  on  the  C3I  FFRDC’s  performance. 

The  Air  Force,  in  its  spring  review,  considers  Service  needs,  ceilings,  all  sources  of  systems 
engineering  and  integration,  and  past  performance  evaluations  to  develop  budget  development 
material.  This  is  reviewed  by  the  Commander,  Air  Force  Materiel  Command  and  approved  by  the 
Assistant  Secretary  of  the  Air  Force,  Acquisition. 
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The  focal  points  for  major  users:  (1)  assemble  and  prioritize  needs  for  MITRE  C3I FFRDC 
support;  (2)  review  requested  work  to  ensure  compliance  with  the  core  work  statement;  and  (3) 
participate  as  members  of  the  JUC. 

The  individual  government  project  officers:  (1)  determine  the  needs  for  C3I  FFRDC  support;  (2) 
provide  funding  for  CJI  FFRDC  support;  and  (3)  define  the  objectives  of  individual  C3I  FFRDC 
projects  in  the  light  of  government  program  objectives,  and  work  with  MITRE  project  leaders  to 
establish  and  adjust  project  plans  and  priorities;  provide  evaluations  of  MITRE  project 
performance  to  MITRE  project  leaders,  managers  of  the  supported  government  programs, 

MITRE  management,  and  the  secondary  sponsors. 

Several  factors  impact  a  program’s  budget  formulation.  The  Department’s  Planning, 
Programming,  Budgeting,  and  Execution  cycle  is  central  to  planning  and  executing  the  funding  for 
each  program  Since  the  CT  FFRDC  is  “industrially  funded,”  programmatic  funding  decisions 
directly  affect  the  amount  of  C3I  FFRDC  support  a  program  can  afford.  Program  managers  and 
MITRE  project  leaders  work  together  to  negotiate  the  appropriate  amount  of  support  to  the 
program.  Once  a  funding  level  is  agreed  upon,  ceiling  must  be  available  in  order  for  the  support 
to  fit  within  the  C31  FFRDC’ s  work  program.  The  availability  of  ceiling  may  also  affect  the 
amount  of  C3I  FFRDC  support. 

MITRE  is  evaluated  quarterly  by  the  Air  Force  and  annually  by  all  other  sponsors. 1  The  C3I 
FFRDC  is  rated  in  several  areas  including: 

1 .  Meeting  T echnical  Needs 

2.  Project  Milestones  and  Schedule 

3 .  Responsiveness  to  Changes 

4.  Project  Staffing 

5.  Managing  Costs 

6.  Value  of  FFRDC 

7.  Summary  Appraisal 

Each  rating  is  on  a  five  category  scale,  ranging  from  excellent  to  poor.  The  Air  Force’s  quarterly 
reviews  supplement  an  annual  assessment.  The  annual  assessment  examines  customer  satisfaction 
for  each  user  and  the  results  are  reviewed  by  the  ESC  Commander  and  the  MITRE  President.  It 
includes  a  six  part  scale. 

An  evaluation  by  the  Army  is  shown  in  figure  F-7. 


1  The  Commander.  Electronic  Systems  Center  reviews  all  projects  being  supported  in  his  System  Program  Offices 
at  these  reviews. 


F-ll 


Figure  F-7:  Army  Performance  Evaluation  Report  Total 
164  respondents  out  of  177 


Percent 

SOURCE:  U.S.  Army 

In  1994,  MITRE’s  work  on  the  Army  contract  was  rated  on  164  different  projects  out  of  177 
projects  and  subtasks.  The  program  ratings  were:  68  percent  excellent,  24  percent  above 
average,  8  percent  average,  1  percent  below  average,  and  one-tenth  of  a  percent  poor.  The 
ratings  by  agency  were  universally  high,  with  ASD/C3I  giving  the  highest  excellent  rating  (86%) 
and  the  Army  providing  the  lowest  (57%),  although  it  provided  no  significant  rating  below 
average  (three-tenths  of  a  percent  below  average,  none  poor).  As  the  same  rating  system  has 
been  used  for  the  last  three  years,  it  is  very  easy  to  see  that  the  overall  ratings  have  improved 
noticeably  since  FY1992  (57%  excellent). 

The  Thoroughness  of  the  Five-Year  Reviews 

The  Comprehensive  Review  for  The  MITRE  C3I FFRDC  conducted  by  the  sponsor  is  lengthy. 
The  criteria  established  by  FAR  35.01 7-4(c)  were  used  to  determine  the  continued  usefulness  of 
MITRE  and  its  effectiveness.  These  are  listed  below. 

•  Examination  of  the  special  technical  needs  and  mission  requirements  that  are  by  the 
MITRE  C3I  FFRDC; 

•  Consideration  of  alternative  sources  to  meet  the  DoD’s  needs; 

•  An  assessment  of  MITRE’s  effectiveness  and  efficiency  in  meeting  the  DoD’s  needs; 

•  An  assessment  of  the  adequacy  of  MITRE’s  management  in  ensuring  a  cost  effective 
operation;  and 

•  Determination  that  the  criteria  for  establishing  MITRE  continue  to  be  satisfied  and  that 
the  sponsoring  agreement  is  in  compliance  with  the  FAR  35.017-1 . 
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The  latest  Comprehensive  Review  was  approved  on  June  1 1,  1993.  It  was  conducted  by  a  C  I 
Task  Force  under  the  auspices  of  ASD(CT). 

The  Review  concluded  that  the  need  for  a  C3I FFRDC  had  not  changed  since  the  establishment  of 
MITRE  in  1958.  The  Review  noted  that  the  1991  conflict  in  the  Persian  Gulf  had  demonstrated 
the  growing  importance  of  the  FFRDC’ s  mission.  The  C3I  budget  was  growing.  The  Review 
stated  the  C3I  FFRDC  provided  a  means  for  the  DoD  to  better  access  commercially  available 
hardware  and  software  The  Center  had  access  to  a  wide  variety  of  systems,  both  military  and 
commercial  Its  special  status  facilitated  interaction  with  industry.  The  Review  outlined  the 
manner  of  work-planning  used  by  the  C3I  community  to  place  work  in  the  C3I  FFRDC,  and  the 
use  of  Industrial  Funding.2  These  approaches  were  cited  as  evidence  the  community  thinks  that 
the  Center  remains  vital  to  meeting  a  growing  DoD  technical  need. 


The  Review  considered  “in-house,”  for-profit,  not-for-profit  (non-FFRDCs)  and  other  FFRDCs 
(DoD  and  non-DoD).  The  discussion  of  alternatives  was  conducted  in  general  terms  with  no 
analysis  conducted  of  specific  alternatives.  At  one  point  the  narration  noted  that  “at  this  time,  it  is 
impossible  to  accurately  estimate  the  cost  of  bringing  all  the  work  performed  by  the  MITRE  C  I 
FFRDC  in-house.”  Further,  the  Review  noted  that  because  in-house  salary  levels  are  low  and  in- 
house  engineering  strength  is  low,  there  was  no  reason  to  consider  in-house  as  a  viable  option. 

For-profit  firms  were  perceived  to  have  conflicts  of  interest.  Further,  citing  a  1961  congressional 
report,  the  Review  stated  that  no  single  firm  was  capable  of  having  the  breadth  of  talent 
necessary.  The  principal  argument  against  other  sources  was  that  each  task  requires  a  sole-source 
certification  that  finds  that  no  other  source  can  do  the  particular  task.  In  this  context,  the  Review 
notes  that  the  alternative  source  issue  is  confronted,  and  justified,  on  a  daily  basis. 

The  assessment  of  the  C3I  FFRDC ’s  efficiency  and  effectiveness  was  based  on:  (1)  an  assessment 
from  high  level  users;  (2)  a  determination  of  the  overall  value  of  services  received;  (3)  a  bottom- 
up  analysis  from  project  managers,  and  (4)  sponsor  and  JUC  assessments.  The  Review  outlines 
the  on-going  evaluation  process.  The  primary  sponsor  also  assembled  the  review  data  for 
FY1990,  FY1991,  and  FY1992  MITRE  projects.  All  major  Air  Force  and  Army  contract 
deliverables  were  reviewed  and  summarized.  These  and  other  summary  reviews  were  included  as 
attachments  of  the  Review.  The  examination  of  the  full  set  of  past  and  current  project  reviews 
lead  to  the  conclusion  that  the  work  conducted  by  the  Center  was  highly  effective  and  efficient 
“particularly  due  to  the  special  relationship  DoD  has  with  MITRE’s  C3I  FFRDC.”  This 
relationship  was  noted  as  a  further  justification  for  the  continued  need  and  use  of  MITRE. 

The  Review’s  assessment  of  management  of  a  cost-effective  operation  states  that  “DoD  has  an 
on-going  process,  both  internal  and  external,  to  assure  the  MITRE  C3I  FFRDC  is  and  remains 
cost  effective,”  and  that  the  process  has  confirmed  that  MITRE’s  management  is  cost 
consciousness.  The  Review  discusses  the  oversight  procedure,  outlining  responsibilities,  and 
noting  DCAA’s  role.  An  attachment  details  MITRE’s  internal  cost  control  activities.  Several 

2  Industrial  Funding  refers  to  the  practice  of  each  user  funding  its  C3I  FFRDC  work  for  various  Program  Elements 
rather  than  a  single  Program  Element  on  Budget  Line. 
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examples  are  given  of  cost  saving  measures.  An  explanation  of  salary  levels  is  provided  and  all 
projects  are  also  evaluated  on  costs.  DoD  concluded  that  no  problems  were  observed  in  this  area 
by  sponsors  during  the  period  under  review. 

Finally,  the  Comprehensive  Review  determined  that  the  original  criteria  for  establishing  the 
MITRE  C31  FFRDC  continues  to  exist  today.  The  primary  criteria  for  an  entity  performing 
systems  engineering  functions  is  that  the  entire  organization  must  not  be  allowed  to  compete  or 
form  business  partnerships  (e  g.,  consortium)  with  any  “for-profit”  organization.  MITRE  enjoys 
this  “trusted  agent”  status  because  the  entire  Corporation  (both  FFRDC  and  non-FFRDC  centers) 
has  never  competed  with  any  “for-profit’s  organization.  To  do  otherwise,  the  Review  contends, 
would  create  a  perceived  conflict  of  interest  and  most  likely  deprive  the  entity  of  access  to 
proprietary  and  privileged  data.  The  Review  goes  on  to  say  that  “the  organization  must  refrain 
from  any  aspect  of  development  or  production,  since  this  would  tend  to  destroy  its  ability  to  be 
technically  objective,  and  that  the  organization  must  be  composed  of  outstanding  scientists  and 
engineers  integrated  into  a  professional  team  that  is  without  peer  in  its  specialized  field.”  Further, 
that  “the  professional  entity  accomplishing  these  functions  must  have  a  close  and  continuing 
relationship  with  the  DoD,  and  it  must  have  a  single-minded,  purposeful  dedication  to  both  the 
DoD  and  its  special  field  of  interest.”  The  Review  concludes  that  the  MITRE  Corporation  and 
the  MITRE  CT  FFRDC  satisfy  all  of  the  above  criteria. 
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Appendix  G 


The  Aerospace  Corporation 

The  written  and  briefing  material  presented  by  the  Aerospace  Corporation  and  its  sponsor  are 
summarized  below.  The  sponsor  was  very  supportive  of  the  role  of  the  Corporation,  the  need  for 
the  work  being  done,  and  the  high  quality  of  that  work. 

The  objective  and  guidelines  for  scope  of  work,  organizational  structure,  size,  and 
appropriateness  of  customers. 

The  Aerospace  Corporation  is  a  private  nonprofit  mutual  benefit  corporation  chartered  in  the 
State  of  California.  It  was  established  in  1960,  and  as  its  primary  activity,  operates  an  FFRDC  in 
support  of  Air  Force  and  national  security  space  programs.  The  FFRDC  sponsor  is  the  Assistant 
Secretary  of  the  Air  Force  for  Acquisition  (SAF/AQ),  with  day-to-day  management  by  the  Air 
Force  Space  and  Missile  Systems  Center  (SMC).  The  Corporation  has  supported  efforts  on 
launch  vehicles  for  Projects  Mercury  and  Gemini,  NASA’s  Viking  and  Voyager  programs,  the 
Ballistic  Missile  Program,  and  the  Space  Shuttle.  The  Corporation  currently  supports  all  DoD 
launch  vehicles.  Air  Force  military  communications  satellites,  weather  satellites,  early-warning 
satellites,  the  Global  Positioning  System  (GPS),  other  National  Security  space  systems,  and 
ballistic  missile  defense.  Participation  in  non-defense  programs,  including  NASA,  has  in  the  past 
been  as  high  as  18%  of  technical  support,  and  was  about  5%  in  FY  95  and  6%  in  FY  96. 
Aerospace  carries  out  work  for  DoD,  NASA,  the  Department  of  Transportation,  other  federal 
and  local  government  agencies,  and  some  foreign  countries  and  international  organizations 

The  Aerospace  Corporation’s  structure  is  shown  in  figure  G-l. 

Aerospace  FFRDC’s  mission  is  to  support  the  Nation’s  national  security  space  efforts,  particularly 
those  executed  by  the  USAF.  The  mission  involves  applying  the  full  resources  of  modem  science  and 
technology  to  achieve  continuing  advances  in  military  space  and  space  related  systems  which  are  basic 
to  national  security;  to  provide  national  security  space  efforts  with  an  organization  which  is  objective, 
possesses  high  technical  competence,  and  is  characterized  by  permanence  and  stability;  to  provide  a 
vital  link  between  the  U.S.  Government  and  the  scientific  and  industrial  organizations  in  the  country 
with  a  capability  and  an  interest  in  the  space  field;  and,  through  its  unique  role,  to  help  to  ensure  that 
the  full  technical  resources  of  the  nation  are  properly  applied  to  developing  highly  reliable  and  cost 
effective  space  and  space  related  systems,  and  that  the  potential  advances  in  the  space  field  are  realized 
in  shortest  possible  time. 


Figure  G-l:  The  Aerospace  Corporation 
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The  Aerospace  FFRDC  is  intimately  integrated  with  the  Space  and  Missile  Systems  Center  (SMC)  and 
other  national  security  space  customers.  Indeed,  SMC  states  that  "Aerospace  is  an  integral  part  of 
SMC."  To  meet  current  SMC  needs,  the  Air  Force  has  divided  a  workforce  roughly  into  thirds:  the 
military  and  government  employees  accounting  for  one-third,  the  Systems  Engineering  and  Technical 
Assistance  (SETA)  contractors  another  third,  and  Aerospace  making  up  the  final  third.  The  SMC 
commander  stated  that  he  has  a  greater  requirement  for  Aerospace  than  is  allowed  under  the  current 
ceiling.  SMC’s  projected  program  requirements  for  Aerospace,  for  example,  are  20  percent  above  the 
available  MTS.  SMC’s  requirements  versus  the  available  support  from  Aerospace  is  shown  in  figure 
G-2. 
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Figure  G-2:  DoD  MTS  Requirements  versus  Available  Aerospace  MTS  Support 


SOURCE:  U.S.  Air  Force 

Aerospace  funding  has  been  declining  (figure  G-3).  It  is  $306  million  for  FY  96.  In  constant  dollars, 
FY  95  funding  was  about  the  same  as  in  FY  82,  although  Aerospace  reports  the  scope  of  space 
programs  supported  is  now  much  larger. 

Figure  G-3:  Aerospace  Funding  Profile  Actual  Obligations 
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SOURCE:  Department  of  Defense 
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The  DoD  remains  the  primary  source  of  funds  for  Aerospace.  In  1995,  of  the  $349.7  million 
Aerospace  received,  only  5%  came  from  non-DoD  sources.  Sixty-two  percent  came  from  SMC, 
another  1%  from  other  Air  Force  offices,  28%  from  "classified"  programs  (it  is  assumed  that  some  of 
this  was  Air  Force  funds)  and  3%  from  other  DoD  sources  (figure  G-4).  Federal  Acquisition 
Regulations  (FAR  35.001)  on  FFRDCs  state  that  the  organizations  will  receive  at  least  70%  of  their 
financial  support  from  the  government  Aerospace  is  far  below  the  30%  limitation  on  non-U.S. 
Government  funding  (currently  less  than  2%),  and  is  not  being  encouraged  to  diversify. 


Figure  G-4:  Aerospace  Technical  Effort  FY  95 


SOURCE:  The  Aerospace  Corporation 

The  Aerospace  staff  has  declined  along  with  the  budget  decline  (figure  G-5).  Non-technical  staff  has 
been  reduced  preferentially.  As  of  November  1995,  Aerospace’s  entire  work  force  had  declined  29% 
from  a  high  of 4,257  in  November  1990  to  3,034.  More  than  1,000  people  were  retired  early  or  laid- 
off  Two-thirds  of  the  current  staff  are  technical  staff  Of  the  technical  staff,  27%  have  Ph.D.  degrees 
and  43%  have  masters.  Only  2%  have  no  degree.  Twenty-four  percent  are  electrical  engineers,  21% 
are  mechanical  and  aeronautical  engineers,  and  21%  are  physicists  and  mathematicians. 
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Figure  G-5:  Aerospace  Corporation  Population 


SOURCE:  The  Aerospace  Corporation 

The  loaded  rate  for  Aerospace  labor  have  also  declined  during  this  period.  In  1995  the  cost  of  an  MTS 
per  year  was  $1 75,000.  The  average  salary  of  MTS  is  $81,500  a  year  while  the  average  salary  of 
support  staff  is  $42,700  a  year.  The  total  cost  per  man-year  at  Aerospace  slowly  and  steadily  declined 
in  real  terms  since  1986.  Overhead  rates  have  been  declining  since  1990. 1 

According  to  the  presentation  of  core  capabilities.  Aerospace  provides  support  not  available  from  the 
USAF’s  in-house  technical  and  engineering  capabilities.  The  Sponsoring  Agreement  states  that  the 
Corporation  shall  perform  scientific  engineering  and  technical  oversight  tasks  in  the  following  areas: 

•  General  Systems  Engineering  and  Integration  (GSE&I) 

•  Technical  Review  (TR) 

•  Selected  Research,  Development,  Test  and  Evaluation  (SRDT&E) 

•  Plans  and  systems  Architecture  (R&SA) 

•  Mission  Oriented  Investigation  and  Experimentation  (MOIE) 

•  Multi-Program  System  Enhancement  (MPSE) 

•  Engineering  Methods  (EM) 

•  Foreign  Technology  Support  (FTS) 


1  The  Aerospace  documentation  shows  that  from  1987  to  1994,  MTS  rates  in  constant  1995  dollars  have  declined  to 
92.7%  of  what  they  were  in  1987. 
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Aerospace’s  specific  core  competencies  are: 

•  Launch  certification 

•  System  of  systems  engineering 

•  Systems  development  and  acquisition 

•  Process  implementation 

•  Technology  application 

The  Air  Force  Sponsor  stated  that  the  criteria  for  core  work  for  Aerospace  is  that  it  is  consistent  with: 

•  Aerospace’s  mission,  purpose,  and  capabilities 

•  The  Air  Force’s  need  for  Aerospace’s  core  competencies 

•  The  FTRDC  special  relationship  between  the  Air  Force  and  Aerospace 

Other  factors  that  relate  to  the  core  work  are  the  special  (long-term)  relationship  between  the  Air 
Force  and  the  Aerospace  Corporation  which  is  characterized  by: 

•  Broad,  deep  knowledge  of  space  technologies  and  systems 

•  Detailed  knowledge  of  all  space  systems,  both  in  development  and  operational 

•  Application  of  processes  for  architectures,  acquisition,  T&E 

•  In-depth  understanding  of  the  operational  role  of  all  space  systems 

•  Substantial  involvement  with  developers  and  users 

The  criteria  for  using  the  Aerospace  FFRDC  according  to  the  Sponsoring  Agreement  are  shown  in 
table  G-l. 


Table  G-l :  Criteria  for  Work  Assignment  to  an  FFRDC 

Freedom  from  bias  for  design,  hardware,  or  approach 

•  Need  for  state-of-the-art  information 

•  Access  to  DoD  planning  information 

•  Access  to  intelligence 

•  Need  for  industry  proprietary  data 

•  Access  to  industry  proposals 

•  Need  for  extensive  background  information 

•  Need  for  diversified  skills 

•  Need  for  outstanding  specialists  in  specific  fields 

•  Continuity  of  effort 

•  Need  for  large  special  facilities 

SOURCE:  U.S.  Air  Force 
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The  FFRDC  reports  that  the  program  manager  defines  tasks  and  certifies  in  writing  to  the  SMC  chief 
engineer  that  the  criteria  have  been  applied  and  validated,  and  that  there  are  no  alternative  government 
or  industry  sources. 

The  customer  breakdown  and  the  work  breakout  for  Air  Force  managed  tasks  for  FY  96  are  shown  in 
figure  G-6. 


Figure  G-6:  Aerospace  MTS  Customers  (FY  96) 


SOURCE:  U.S.  Air  Force 

The  Sponsor  stated  that  the  Aerospace  FFRDC  has  no  non-core  work. 

The  appropriateness  of  non-DoD  work,  and  non-sponsored  work  undertaken  by  the  FFKDCs  and 
UARCs. 

Aerospace  can  do  non-sponsor  and  non-DoD  work.  There  is  currently  about  3%  non-sponsor  DoD 
work  and  6%  non-DoD  work  for  FY  96.  The  sponsor  reports  that  government  and  commercial  work 
is  approved  by  SMC.  Commercial  work  must  be  in  the  national  interest,  must  be  non-competitive  with 
industry  capabilities  and  must  not  create  any  reasonable  perception  of  potential  conflict-of-interest. 

The  justification  for  all  work  must  address: 

•  Aerospace  unique  qualifications 

•  A  sole  source  justification  based  on  the  sponsoring  agreement 

•  The  benefit  to  SMC, .  the  Air  Force,  the  Nation 

•  The  requesting  organization  and  its  present  relationship  to  SMC 
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Some  examples  are  shown  in  table  G-2. 


Table  G-2:  Examples  of  Non-Sponsor  Work 


•  NASA  (52%) 

—  Space  Station,  Hubble,  Cassini 

•  NOAA  (13%) 

-  NPOESS 

•  International  (6%) 

—  CNES  Ariane  5 

—  Norway  -  launch  site  environmental  assessment 

-  Korea  -  space  engineering  training 

Source:  The  Aerospace  Corporation 


Compliance  with  the  DoD  Management  Plan 

The  DoD  Management  Plan  states  that  annual  levels  of  effort  will  be  based  on  the  application  of  the 
core  concept  and  guideline  fee  for  Systems  Engineering  and  Integration  Centers  that  include:  (1) 
maintaining  stable  core  competencies,  and  (2)  responding  to  projected  trends  in  workload  and  funding 
consistent  with  the  budget  supporting  the  mission  area. 

The  Plan  states  that  DoD  FFRDCs  may  only  perform  core  work,  that  such  work:  must  be  approved 
by  the  Primary  Sponsor;  can  only  be  accomplished  for  DoD,  other  government  entities  (including  state 
and  municipal  governments)  and  for  not-for-profit  activities;  cannot  include  commercial  woric. 

Parent  institutions  operating  DoD  FFRDCs  may  perform  non-FFRDC  work,  but  such  work  must 
conform  to  some  specific  guidelines:  be  in  the  national  interest;  not  undermine  the  independence, 
objectivity  and  credibility  of  the  institute,  or  call  into  question  the  freedom  from  conflict  of  interest;  not 
take  unfair  advantage  of  FFRDC  work;  and  normally  work  in  the  public  sector,  although  commercial 
work  may  be  accepted  if  the  sponsor  agrees. 

Fees  may  be  authorized.  The  Plan  calls  for  “an  annual  fee  proposal  justifying  each  element  of  fee.” 

The  FFRDC  and  the  sponsor  both  stated  that  the  FFRDC  is  currently  capable  of  maintaining  core 
competencies  in  critical  areas,  but  neither  is  sure  they  can  continue  to  do  so  in  the  fixture.  The  sponsor 
noted  a  gap  between  MTS  required  and  MTS  available  under  the  current  congressional  ceiling— 2, 1 74 
MTS  required,  and  1,765  MTS  delivered  in  FY  1996. 

Figure  G-2  indicates  that  the  MTS  decline  has  been  less  steep  than  the  overall  dollar  decline  and  that 
much  of  the  personnel  decline  has  come  in  the  support  staff 
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The  sponsor  provided  a  decision  tree  for  task  allocation  that  focuses  Aerospace  work  on  areas  that 
cannot  be  met  by  either  other  industry  or  government  organizations,  and  are  therefore  core  work  (see 
figure  G-7). 


Figure  G-7:  Decision  Tree  for  Task  Allocation 


SOURCE:  U.S.  Air  Force 

The  criteria  for  selection  of  projects,  discussed  earlier,  also  help  limit  work  to  that  considered 
appropriate  for  Aerospace.  Non-Sponsor,  non-core  work  can  be  accepted  by  Aerospace,  but  outside 
the  FFRDC,  and  only  after  the  limits  of  such  work  have  been  negotiated  with  the  sponsor. 
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Sponsors  Management  Process 

The  Air  Force  Sponsoring  Agreement  details  the  management  responsibilities  of  both  Aerospace  and 
the  Air  Force  Sponsor.  Day-to-day  Air  Force  management  is  the  responsibility  of  the  Air  Force  Space 
and  Missile  Systems  Center  (SMC).  SMC  is  responsible  for  setting  the  broad  program  direction; 
oversight  of  ongoing  development  programs;  review  of  annual  work  plans  and  manpower  allocation; 
recurrent  assessment  of  Aerospace  FFRDC  efficiency  and  effectiveness;  evaluation  of  the  work 
performed  by  the  Aerospace  FFRDC;  and  resolution  of  the  intraprogram  conflicts  involving  the 
Aerospace  FFRDC. 

Figure  G-7  outlined  how  task  allocation  is  made  SMC  has  a  system  to  evaluate  Aerospace’s  work. 
According  to  SMC,  customer  satisfaction  ratings  are  consistently  "superior."  SMC  says  that  the  most 
common  complaint  is  "I  can't  get  all  the  Aerospace  MTS  I  need!".  Beyond  these  obvious  statements  of 
satisfaction,  there  is  an  semi-annual  evaluation  system  based  around  the  award  fee.  In  the  last  four 
years  (FY  91  to  FY  95),  from  70%  to  more  than  90%  of  Aerospace's  work  has  been  rated  above 
standard  and  30%  to  10%  is  rated  as  meeting  standard,  with  a  generally  increasing  trend  in  the  fraction 
of  work  rated  “above  standard.”  No  work  has  been  rated  as  below  standard  since  FY  90.  (See  figure 
G-8). 


Figure  G-8:  Semi  Annual  Evaluation  of  Aerospace  Performance  by  Customers 
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SOURCE:  The  Aerospace  Corporation 

The  ranking  system  evaluates  the  following  areas  as  either  "above  standard",  "meets  standard",  or 
"below  standard": 
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1 .  Effectiveness  of  Aerospace  Management  Approach 

2.  Problem  Solving  Ability 

3 .  Responsiveness  to  Program  Needs 

4.  Adequacy  of  Aerospace  Support 

5.  Technical  Competence  and  Objectivity 

6.  Initiative  and  Cooperations  of  Supporting  Team 

7a.  System  Program  Office  (SPO)/Aerospace  Working  Relations,  Management  Level 
7b.  System  Program  Office  (SPO)/Aerospace  Working  Relations,  Working  Level 
8a.  Work  Force  Capability,  Key  people 
8b.  Work  Force  Capability,  Supporting  MTS 

9.  Visibility  of  Aerospace  Support 

10.  Technical  Accomplishments 

1 1 .  Overall  Quality  of  Aerospace  Effort 

As  a  part  of  the  management  structure,  the  award  fee  is  set  up  to  provide  an  incentive  to  managers  to 
“enhance  the  technical  and  cost  effectiveness  of  the  FFRDC.”  Furthermore,  Aerospace  stated  that  it 
does  have  a  clear  quantitative  and  comparative  measure  of  its  overall  performance:  the  success  of  its 
launches  compared  to  U.S.  commercial  launches,  and  the  early  life  success  records  of  satellites. 

The  Thoroughness  of  the  Five-Year  Reviews 

The  Comprehensive  Review  for  the  Aerospace  Corporation  conducted  by  the  sponsor  is  lengthy. 
The  criteria  established  by  FAR  35.017  were  used  to  determine  the  continued  usefulness  of 
Aerospace  and  its  effectiveness.  These  are  listed  below. 

•  Examination  of  the  technical  needs  and  mission  requirements  that  are  served; 

•  Consideration  of  alternative  sources  to  meet  the  DoD’s  needs; 

•  An  assessment  of  Aerospace’s  effectiveness  and  efficiency  in  meeting  the 
DoD’s  needs; 

•  An  assessment  of  the  adequacy  of  Aerospace’s  management  in  ensuring  a  cost 
effective  operation;  and 

•  Determination  that  the  criteria  for  establishing  Aerospace  continues  to  be 
satisfied  and  that  the  sponsoring  agreement  is  in  compliance  with  the  FAR 
35.017-1. 

The  latest  Comprehensive  Review  is  dated  October  27, 1993. 

The  Review  examined  the  long-term  needs  of  the  Air  Force  for  general  engineering  support  of  military 
space  systems.  The  Review  considered  both  non-classified  and  classified  systems.  It  noted  the  risks 
and  costs  associated  with  military  space  and  discussed  the  specific  programs  that  continue  to  need 
support.  Some  of  these  are  new  systems  requiring  Aerospace  to  stay  on  the  cutting-edge  of 
technology.  Others  are  legacy  systems  that  require  corporate  knowledge  that  Aerospace  retains.  The 
Review  of  need  also  addressed  the  potential  of  segregating  out  portions  of  the  overall  military  space 
program,  and  reported  that  such  segregation  created  both  technical  and  conflict  of  interest  problems. 
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The  Review  looked  at  some  specific  alternative  sources.  It  stated  that  the  best-qualified  in-house 
alternative  was  NASA.  The  Review  used  a  recent  (1990)  NASA  capabilities  review  to  analyze  this 
alternative.  The  Review  also  considered  the  Air  Force’s  Phillips  Laboratory.  Neither  NASA,  nor  the 
Laboratory  were  believed  to  be  adequate  alternatives. 

For-profit  firms  were  examined  in  less  detail.  The  analysis  centered  on  problems  in  the  for-profit 
industry — ’’financial  interests,  conflict  of  interest,  and  lack  of  communications”  were  judged  to  have 
detrimental  effects  on  programs.  The  analyses  also  used  estimated  cost-savings  from  Aerospace 
analysis  of  contractor  programs  to  show  the  need  for  objective  analyses.  For-profit  firms  were 
dismissed  for  reasons  of  conflict-of-interest,  objectivity,  independence,  and  stability  and  lack  of  a 
sufficient  range  in  technical  capabilities. 

Non-profit  organizations  were  considered  (including  other  FFRDCs)  but  these  were  deemed  to  be  too 
expensive  and  lacking  in  the  critical  skills.  The  assessment  of  efficiency  and  effectiveness  is  relatively 
short  (7  pages).  The  evaluation  is  based  on  a  survey  of  1 30  individual  users/projects.  The  Review 
discussed  the  long-term  evaluation  system  and  its  use  by  Aerospace  and  Air  Force  management.  The 
Review  also  noted  the  excellent  launch  success  rate  for  missions  supported  by  Aerospace,  and  the 
estimated  cost-savings  involved. 

The  Review  of  management  controls  to  ensure  cost  effective  operations  outlined  corporate 
organization  to  oversee  costs,  and  discussed  overall  auditing  and  management  controls.  It  reviewed  a 
number  of  specific  actions  taken  to  increase  the  cost  effectiveness  of  operations  and  noted  that  cost  per 
MTS  had  been  felling  steadily  in  real  terms  since  1986. 
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Appendix  H 


Software  Engineering  Institute,  Carnegie-Mellon  University 

The  materia]  presented  by  the  Software  Engineering  Institute  (SEI)  and  its  Primary  Sponsor  are 
summarized  in  this  appendix.  The  sponsor  presented  a  very  supportive  view  of  the  need  for  the 
type  of  work  that  is  being  done  by  the  FFRDC  and  the  value  of  SEI’s  products. 

Objectives  and  guidelines  for  scope  of  work,  organizational  structure,  size ,  and 
appropriateness  of  customers. 

The  Software  Engineering  Institute  was  established  in  1984  at  Carnegie  Mellon  University 
through  a  competitive  procurement  process  initiated  by  the  DoD  with  the  approval  of  Congress. 
The  Center  was  proposed  as  a  response  to  the  rapid  growth  of  mission-critical  software  systems 
being  put  into  DoD  equipment  that  were  a  growing  element  of  the  budget  and  taking  an 
increasingly  large  element  of  technical  capability  to  support  them.  A  DoD  Joint  Service  Task 
Force  chartered  to  review  embedded  software  problems  proposed  the  establishment  of  a  Software 
Engineering  Institute  to  bridge  the  gap  between  R&D  activities  and  the  exploitation  of  those 
techniques  in  systems.  A  subsequent  report  by  IDA  validated  the  need  for  such  an  organization 
and  recommended  that  the  Institute  be  an  FFRDC. 

The  SEI’s  mission  is  to:  (1)  bring  the  most  effective  technology  to  bear  on  rapid  improvement  of 
the  quality  of  operational  software  in  DoD  software  intensive  systems;  (2)  accelerate  the 
reduction  to  practice  of  modem  software  engineering  technology;  (3)  promulgate  the  use  of  this 
technology  throughout  the  software  community;  and  (4)  foster  standards  of  excellence  for 
improving  software  engineering  practice. 

The  Defense  Advanced  Research  Projects  Agency  (DARPA)  is  the  primary  sponsor.  The  Air 
Force  Electronic  System  Command  (ESC)  serves  as  the  Administrative  Agent. 

SETs  structure  is  shown  in  figure  H-l . 


Figure  H-l:  Software  Engineering  Institute 


SOURCE:  Software  Engineering  Institute 

Table  H-l  shows  funding  trends  over  the  last  four  years.  The  FY  96  budget  is  $33.3  million. 


Table  H-l:  Income  by  Source 


($M)  FY  1992 

Total  DoD  Funds  34.9 

Total  Non-DoD 

Total  SEI  Revenue  35.07 

DoD  Ceiling  35.07 


FY  1993 

FY  1994 

FY  1995 

31.8 

30.6 

28.7 

.5 

2.3 

3.1 

32.4 

32.9 

31.8 

36.24 

35.8 

28.7 

SOURCE:  Software  Engineering  Institute 

SEI  had  a  staff  of  184  total  MTS  in  1995  (166  DoD  MTS,  18  non-DoD  MTS).  Personnel  trends 
are  shown  in  figure  H-2. 
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Figure  H-2:  SEI  Personnel  History 


SOURCE:  Software  Engineering  Institute 

Twenty-eight  percent  of  the  research  staff  held  doctorates,  40%  Masters,  27%  Bachelor  s  and  5% 
lacked  a  college  degree.  About  65%  of  the  staff  came  from  industry. 

The  cost  of  MTS  have  been  relatively  stable  and  has  fallen  slightly  in  real  terms  over  the  past 
three  years.  SEI  notes  that  15%  of  MTS  labor  is  provided  through  an  affiliate  program  at  no  cost 
to  the  government,  but  SEI  still  provides  support  for  that  labor  so  tyTS  expenditures  appear 
higher  than  they  actually  are. 

The  SEI  reports  that  it  possesses  core  competencies  in  software  engineering  and  the 
supporting  software  technology,  and  technology  transition. 

Software  engineering  and  the  supporting  software  technology  is  rapidly  evolving.  The  SEI 
technical  program  currently  encompasses  work  in  four  broad  areas  of  interest  and  significance  to 
the  software  engineering  community:  software  process,  risk  management,  disciplined 
engineering,  and  trustworthy  systems. 

Software  process  focuses  on  software  process  improvement  based  on  process  modeling, 
representation,  measurement,  and  supporting  technology  to  define  and  manage  processes, 
whether  automated  or  human  directed.  Risk  management  focuses  on  defining  and  modeling 
processes  for  assessing  and  managing  software  development  risks.  Disciplined  engineering 
focuses  on  methods  and  tools  for  engineering  software  systems.  Trustworthy  systems  focuses 
on  the  importance  of  trustworthy  software,  computer  systems,  and  their  potential  vulnerability  to 
disruptive  or  otherwise  criminal  activities. 
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Technology  transition  aims  at  encouraging  the  use  of  best  practices  throughout  the  software 
community.  This  includes  enabling  activities  to  mature  the  infrastructure  of  the  software 
engineering  profession  and  transition  software  engineering  strategies  and  methods.  Maturing  the 
infrastructure  focuses  on  establishing  and  improving  the  structures  needed  to  support  a  software 
engineering  profession  with  the  expectation  that  such  an  infrastructure  assists  in  disseminating 
improved  software  engineering  practices.  Transition  strategies  and  methods  activities  focus  on 
ensuring  that  effective  software  engineering  technology  transition  practices  are  used  by  the  entire 
software  engineering  community. 

SEI  will  not  accept  non-core  work.  SEI  reports  that  core  work  is  defined  by  the  combination  of 
SEI’s  purpose,  mission,  and  capabilities,  SEI  core  competencies,  and  the  strategic  relationship 
maintained  by  SEI  and  its  sponsors. 

To  be  consistent  with  the  purpose,  mission,  and  capabilities,  work  should  be: 

•  targeted  to  bringing  an  engineering  discipline  to  the  way  software  is  acquired, 
developed,  and  maintained. 

•  targeted  toward  transitioning  the  use  of  software  engineering  and  supporting 
technology. 

•  involve  work  to  research,  evaluate,  mature,  or  transition  promising  technology. 

The  work  must  be  consistent  with  SEI  core  competencies  of  software  engineering,  supporting 
software  technology,  or  technology  transition. 

The  work  should  require  one,  or  a  combination  of,  the  following  characteristics. 

•  knowledge  of  sponsors’  needs  and  problems. 

•  knowledge  of  the  technical  opportunities  that  will  address  sponsors’  needs  and 
problems. 

•  objectivity  from  real  or  perceived  biases  or  conflicts  of  interest. 

•  demonstrated  ability  to  protect  proprietary  data  that  industry  does  not  give 
freely. 

•  ability  to  protect  information  sensitive  to  government  acquisitions. 

•  ability  to  provide  in-depth  knowledge  of  sponsor’s  programs  and  operations. 

•  high  quality  and  value  of  work. 
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relationship  with  and  ability  to  influence  the  software  engineering 
infrastructure. 


The  work  must  be  sponsored  by  an  appropriate  organization.  SEI  may  accept  work  from:  the 
DoD,  other  federal  agencies,  state  or  local  governments,  and  nonprofit  organizations.  The  SEI 
may  enter  into  Cooperative  Research  and  Development  Agreements  (CRADA)  with  industry, 
subject  to  DARPA  approval. 

As  noted  above,  all  work  is  core  work.  SEI  reports  that  in  1995  53%  of  its  work  was  for  DoD 
exploratory  R&D,  37%,  DoD  development  R&D,  and  10%  non-DoD — much  of  the  non-DoD 
work  is  for  the  FAA. 

The  le\>el  and  appropriateness  of  non-DoD  work  and  work  for  other  than  the  primary  sponsor. 

In  FY  95,  about  1 0%  of  the  work  was  for  non-DoD  customers — primarily  the  FAA.  All  such 
work  is  approved  by  the  sponsor.  This  work  is  considered  core  work. 

Compliance  with  the  DoD  Management  Plan 

The  DoD  plans  directs  Research  and  Development  (R&D)  Laboratories  to  maintain  expertise  and 
related  competencies  necessary  to  address  the  core  work  and  priorities  of  the  sponsor. 

More  generally,  the  Centers  are  directed  to  perform  only  core  work,  and  such  work  must  be  approved 
by  the  primary  sponsor;  can  only  be  accomplished  for  DoD,  other  government  entities  (including  state 
and  municipal  governments),  and  for  not-for-profit  activities.  It  cannot  include  commercial  work. 

Parent  institutions  operating  DoD  FFRDCs  may  perform  non-FFRDC  work,  but  such  work  must 
conform  to  some  specific  guidelines:  be  in  the  national  interest;  not  undermine  the  independence, 
objectivity,  and  credibility  of  the  institute  or  call  into  question  the  freedom  from  conflict  of  interest;  not 
take  unfair  advantage  of  FFRDC  work;  and  normally  work  in  the  public  sector,  although  commercial 
work  may  be  accepted  if  the  sponsor  agrees. 

Fees  may  be  authorized.  The  Plan  calls  for  “an  annual  fee  proposal  justifying  each  element  of  fee.” 

The  SEI  has  complied  with  the  requirements  of  the  DoD  Management  Plan.  Additionally,  the  SEI 
offers  the  following  comments: 

SEI  expressed  concern  that  the  congressional  limits  stopped  the  SEI  planned  and  managed  growth 
before  it  was  able  to  mature.  The  original  ceiling  of 250  software  engineers,  which  was  never 
reached,  has  been  reduced  to  155.  This  represents  an  actual  reduction  of  20%  from  the  1993 
staffing  level  of  1 92 — a  loss  of  nearly  40  experienced  professionals. 

SEI  states  that  it  does  only  core  work  and  that  all  work  is  approved  by  the  sponsor.  SEI  does 
work  with  industry  as  a  part  of  CRADAs.  SEI  states  that  the  work  it  does  with  non-DoD 
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customers  (including  industry)  is  core  work  that  would  appear  to  conform  with  the  spirit  of  the 
DoD  Management  Plan  directive.  There  is  an  elaborate  procedure  of  oversight  of  the  work  that 
SEI  does.  The  sponsor  reviews  and  approves  all  technical  program  plans.  Costs  are  reviewed 
twice  each  year. 

As  an  educational  institution,  Carnegie  Mellon  University  is  excluded  from  the  stricter 
interpretations  of  limitations  on  the  work  that  parent  institutions  may  undertake. 

The  SEI  operates  under  a  cost  reimbursement,  no  fee  contract. 

Sponsor’s  Management  Process 

A  graphical  representation  of  SEI  Management  Oversight  is  shown  at  figure  H-3. 


Figure  H-3:  SEI  Management  Oversight 
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SOURCE:  Software  Engineering  Institute 

DARPA  is  the  Primary  Sponsor.  ESC  is  Administrative  Agent  for  DARPA.  There  is  an  on-site 
JPO  to  provide  day-to-day  administration.  DARPA  provides  strategic  guidance  for  basic  R&D 
(about  53%  of  the  work),  DARPA  provides  general  guidance  for  TO&P  tasking  (about  47%  of 
the  work).  ESC  monitors  the  implementation  of  this  guidance. 

Additional  oversight  is  provided  by  a  Joint  Advisory  Committee  and  a  Senior  Technical  Review 
Group  (STRG).  The  JAC  is  responsible  for  advising  DARPA  in  several  areas  including:  SEI 
strategy,  and  current  and  proposed  programs.  The  JAC  also  is  responsible  for  assisting  SEI  in 
implementing  policies.  The  STRG  provides  technical  advice  to  the  JAC  and  DARPA  on  SEI 
technical  objectives  and  priorities  and  helps  review  programs  plans  and  progress.  SEI  also  uses 
advisory  and  working  groups  to  provide  customer  guidance  and  technical  reviews  of  projects.  A 
Financial/Program  Management  Review  is  held  three  times/year.  A  detailed  technical  review  is 
held  four  times/year. 

SEI  activities  are  also  reviewed  by  its  Board  of  Visitors  and  by  program  technical  advisory  boards 
and  working  groups  that  include  representatives  of  both  government  and  industry. 


H-6 


SEI  operates  under  a  two-year  operating  plan  and  a  five-year  strategic  plan  that  is  approved  by 
DARPA.  Tasks  are  documented  and  approved  by  ESC.  Budgets  are  based  on  approved  work  in 
the  Strategic  Plan.  Budgets  are  prepared  annually  and  updated  every  2  months. 

SEI  work  is  evaluated  annually  by  customers.  The  metrics  for  evaluation  include: 

•  Expectations  met 

•  Timeliness 

•  Usefulness 

•  Value/Cost 

•  Innovativeness 

•  Overall  performance 

The  sponsor  feedback  process  includes  responses  from  customers,  and  interaction  with  SEI  to 
ensure  implementation  of  sponsor  recommendations. 

The  Thoroughness  of  the  Five-  Year  Review 

The  latest  Comprehensive  Review  was  completed  in  1994.  It  addresses  the  six  areas  required  by 
FAR  35.017  and  the  DoD  Management  Plan. 

An  independent  Blue  Ribbon  Panel  chartered  by  DARP A  produced  a  Report  which  provided 
input  to  DARPA’s  Comprehensive  Review.  The  Review  concluded  that  the  DoD  faces  major 
challenges  in  reducing  costs  of  military  systems;  that  the  state-of-practice  in  software  engineering 
within  the  DoD  community  is  far  behind  the  state-of-the-art  in  the  commercial  world;  that  SEI 
had  made  major  contributions  to  improving  the  DoD  practices  and  that  the  growing  need  (some 
software  is  more  complex  and  more  costly)  validated  the  continued  requirement  for  SEI. 

The  Review  examined  the  increasing  importance  of  embedded  software,  and  the  growing  portion 
of  the  DoD  budget  devoted  to  software  (expenditures  have  grown  from  $7  billion/year  to  $35 
billion/year  over  the  last  decade).  The  Review  noted  a  need  to  accelerate  the  transfer  of 
technology  and  a  need  to  develop  improved  management  of  software  problems.  It  outlined 
several  problems  that  SEI  had  helped  solve  and  discussed  the  continued  usefulness  of  the 
organization.  The  Review  noted  that  “the  DoD’s  technical  needs  and  mission  requirements  in 
software  engineering  still  exist,  and  if  anything  are  even  more  challenging  than  ever. 

The  analysis  of  alternative  sources  included  discussions  of:  (1)  government  organizations,  (2) 
industry  organization,  (3)  other  universities,  and  (4)  other  FFRDCs.  The  analysis  of  possible 
government  sources  centered  on  a  discussion  of  SEI  staff.  The  Review  noted  that  SEI  hired 
personnel  with  extensive  industry  experience.  Government  was  unlikely  to  be  able  to  have  that 
experience.  Further,  there  is  no  government  agency  charged  with  software  engineering 
technology  transition. 

The  analysis  of  industry  alternatives  was  general  in  nature.  It  included  discussions  of  projects  in 
which  SEI  was  able  to  work  with  several  competing  firms  (software  for  flight  simulators).  The 
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Review  noted  that  proprietary  and  competitive  concerns  would  inhibit  industry  from  sharing  their 
solutions  to  technical  problems. 

The  analysis  of  other  universities  noted  that  others  might  be  able  to  play  the  same  role,  but  judged 
the  start-up  costs  to  be  excessive.  Here  the  Panel  noted  that  SEI  had  been  established  through  a 
competitive  process.  The  Panel  stated  that  it  did  not  believe  that  re-competition  was  in  the  best 
interest  of  government  because  of  expected  costs. 

Other  FFRDCs  were  also  briefly  considered  and  rejected  because  they  are  generally  focused  on 
other  work. 

The  analysis  ended  with  a  discussion  of  the  uniqueness  of  SEI:  the  SEI  is  an  objective  broker  of 
information;  is  able  to  keep  proprietary/sensitive  information;  is  an  unbiased  source;  has  critical 
mass  of  top  caliber  software  professionals;  provides  a  central  repository  for  information;  has 
necessary  core  competencies  in  critical  software  technologies. 

The  Review  had  an  extensive  discussion  of  SET  s  effectiveness  and  efficiency  in  meeting  the 
sponsor’s  needs.  It  reported  a  number  of  specific  costs  savings  flowing  from  SEI  work  on 
specific  projects.  It  outlined  work  with  organizations  on  the  transfer  of  technology,  noted  the 
advisory  boards  and  working  groups  that  SEI  participated  in  and  its  education  initiatives. 
Judgments  on  efficiency  were  based  on  the  results  of  survey  responses  of  customers.  SEI  had 
good  scores  on  these  surveys — including  judgments  on  costs. 

The  assessment  of  the  management  of  the  SEI  to  ensure  a  cost-effective  operation  discussed  the 
processes  of  DARPA  and  ESC  to  monitor  costs  and  ensure  that  performance  is  commensurate 
with  these  costs.  As  noted  above,  costs  are  a  part  of  the  overall  assessment  done  by  each  project 
sponsor.  These  data  are  tabulated  and  examined  by  the  principal  sponsor.  The  Review  noted  the 
survey  asks  for  both  a  rating  of  overall  “costs  incurred  for  work  done,”  and  whether  the  projects 
were  implemented  in  a  cost-effective  way.1  The  Review  outlined  a  number  of  processes  that  SEI 
and  its  parent  institution  were  conducting  to  ensure  a  cost-effective  operation — monitoring  salary 
increases,  reducing  travel  costs,  and  other  actions,  including  internal  financial  reporting  and 
reviews. 

The  Review  also  noted  the  SEI  leveraged  its  technical  staff  and  used  its  resident  affiliate  program 
to  increase  the  research  staff  effort. 

Finally,  the  Review  listed  the  Joint  Program  Office,  Office  of  Naval  Research,  and  DCAA  reviews 
of  the  cost  effectiveness  of  the  SEI  and  CMU. 


1  Responses  to  the  “costs  incurred  for  work  done”  questions  consistently  received  a  high  percentage  of  acceptable 
answers. 
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Appendix  I 


MIT  Lincoln  Laboratory 

The  material  that  Lincoln  Laboratory  and  its  primary  sponsor  presented  to  the  Task  Force,  along 
with  information  from  additional  sources,  is  summarized  in  this  appendix.  The  Sponsor  was  very 
supportive  of  the  need  for  the  work  conducted  by  Lincoln  Laboratory  and  the  quality  of  that 
work. 

OVERVIEW 

Objectives  and  guidelines  for  scope  of  work,  organizational  structure,  size,  appropriateness  of 
customers. 

Lincoln  Laboratory  is  the  largest  DoD  FFRDC  laboratory.  Project  Lincoln  was  established  in 
1951  at  the  joint  request  of  the  Air  Force,  Navy,  and  Army  following  the  Project  Charles  study, 
and  directed  to  develop  a  prototype  continental  air  defense  against  Soviet  air  attack.  According 
to  the  sponsoring  agreement,  the  Laboratory’s  current  mission  “is  to  carry  out  a  program  of 
research  and  development  pertinent  to  national  defense  with  particular  emphasis  on  advanced 
electronics.”  The  Air  Force  is  the  primary  sponsor  and  provides  the  largest  percentage  of  the 
work  of  the  laboratory.  Contracting  is  primarily  accomplished  through  the  Air  Force  Materiel 
Command’s  Electronics  Systems  Center. 

Lincoln  Laboratory  is  operated  by  MIT  which  has  management  oversight  and  participates  in 
mutual  research  activities  with  Lincoln  Laboratory.  MIT  management  provides  general  policy, 
financial  accountability,  and  reviews  Laboratory  activities.  MIT  has  never  received  a  fee  for  the 
operation  of  Lincoln  Laboratory.  A  DoD  Joint  Advisory  Group  reviews  and  approves  the 
Laboratory  program  annually. 

The  Laboratory  structure  is  shown  in  figure  1-1 . 


Figure  1-1 :  MIT  Lincoln  Laboratory 


SOURCE:  Lincoln  Laboratory 

Figure  1-2  shows  Lincoln  Laboratory’s  DoD  funding.  The  FY  1995  DoD  funding  ceiling  was 
$274.9  million,  a  decline  in  real  terms  of  about  41%  since  FY  1990.  In  constant  1995  dollars,  Lincoln 
Laboratory’s  DoD  funding  is  back  down  to  about  1982  level. 
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Figure  1-2:  Lincoln  Laboratory  Funding  Profile 
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SOURCE:  Lincoln  Laboratory 

DoD  is  the  primary  source  of  funds  for  Lincoln  Laboratory.  DoD  funding  in  1994  comprised  84%  of 
Lincoln  Laboratories  funding.  The  breakout  of  DoD  money  is  shown  in  table  1-1 . 

Table  1-1:  MIT  Lincoln  Laboratory 
FY  95  Budget  vs.  FY  96  (est.)* 


Sponsor  _ $M 


(No.  of  Programs) 

FY  95 

FY96  (1 

Air  Force  (46) 

83 

76 

Navy  (22) 

24 

17 

Army  (22) 

63 

55.5 

DARPA  (37) 

35 

34 

BMDO  (3) 

8 

9 

Special  (6) 

45 

33 

DoD  Line  (9) 

15 

16 

Other  DoD  (6) 

6 

_9 

Total  DoD  (151) 

274(1) 

250 

(1)  Includes  $5M  Deobligations 
*  By  Sponsor  and  Number  of  Programs. 


SOURCE:  U.S.  Air  Force 
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Lincoln  Laboratory’s  major  civilian  customers  are  the  Federal  Aviation  Administration  (FAA)  and  the 
National  Oceanic  and  Atmosphere  Administration.  The  Center  also  works  directly  with  industry  in 
Cooperative  Research  and  Development  Agreements  (CRADA).  There  is  a  limitation  in  the 
sponsoring  agreement  that  the  total  non-U.  S.  Government  funding  (including  CRADAs)  will  not 
exceed  30  percent  of  the  total  laboratory's  funded  support. 

Lincoln  Laboratory  employees  number  approximately  1,800  on  DoD  programs.  The  1995  MTS  was 
1,018.  The  new  DoD  Plan  envisions  a  total  of 920  STE  for  Lincoln.  Manpower  trends  are  shown  in 
figure  1-3. 


Figure  1-3  :  Lincoln  Laboratory  Employment  Trends 


FY 


SOURCE:  Lincoln  Laboratory 

The  Laboratory  reported  that  its  personnel  are  the  key  to  success.  The  composition  of  the  technical 
staffby  degree  and  academic  discipline  is  shown  in  figure  1-4.  The  Laboratory  counts  the  interaction 
with  the  students,  staff,  and  faculty  of  MIT  as  one  of  its  strengths. 
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Figure  1-4:  Composition  of  Technical  Staff 
Doctorate  55%,  Masters  29%,  Bachelors  13%,  No  Degree  3% 


SOURCE:  Lincoln  Laboratory 

The  Laboratory  told  the  Task  Force  that  a  great  deal  of  effort  is  expended  on  personnel  in  ensuring  top 
hires  and  constantly  outplacing  personnel.  The  total  annual  turnover  rate  at  Lincoln  Laboratory  is 
around  5  percent.  Lincoln  Laboratory  has  a  performance  ladder  ranking  system  for  its  technical  staff, 
and  states  that  weak  performers  are  asked  to  leave. 

Lincoln  Laboratory’s  core  work  is  research  and  development  across  the  range  of  electronic  technology 
for  surveillance,  identification,  and  communications;  with  particular  emphasis  on  the  application  of 
these  technologies  to  national  defense  problems  of: 

•  Location,  tracking,  characterization,  analysis  and  identification  of  moving  and  fixed 
targets  by  radar,  optical,  acoustic,  and  other  means; 

•  Communications  at  all  operational  levels  which  are  reliable,  secure,  and  survrvable, 
including  tactical,  long-range,  and  strategic;  and 
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•  Development  of  advanced  electronic  devices,  including  materials,  components, 
subsystems,  and  prototype  fabrication. 

The  work  in  these  areas  spans  systems  conception,  technology  development  and  hardware 
demonstration,  technology  transfer  to  industry,  independent  expert  consulting  and  quick  reaction 
assistance. 

In  carrying  out  its  core  work,  Lincoln  Laboratory  organizes  its  activities  into  different  mission  areas. 
These  have  evolved  over  time  and  are  expected  to  change  in  the  future.  Current  mission  areas  include: 

Ballistic  Missile  Defense :  Research  and  development  related  to  ballistic  missile  defense  and  offense 
technologies. 

Communications:  Research  and  development  related  to  short-  and  long-range  military 
communications  systems,  including  space-,  air-,  shipboard-  and  ground-based  terminals. 

Space  Control:  Research  and  development  related  to  satellite  surveillance  and  identification  from 
Earth-based  sensors,  and  surveillance  of  targets  moving  on  or  near  the  surface  of  the  earth  from  space. 

Air  Defense:  Research  and  development  related  to  air  defense  activities. 

Surface  Surveillance:  Research  and  development  related  to  battlefield  surveillance  and  target 
detection,  signal  intercept  techniques,  and  tactical  communications. 

Advanced  Electronics  Technology:  Research  and  development  related  to  advanced  electronics  for 
application  to  defense  programs. 

The  breakout  of  the  current  focus  of  specific  work  in  these  general  areas  is  shown  in  table  1-2. 


Table  1-2  :  MIT  Lincoln  Laboratory  Mission  Objectives/Future  Focus 


MISSION 

AREAS: 

BALLISTIC 

MISSILE 

DEFENSE 

AIR  DEFENSE 

SURFACE 

SURVEILLANCE 

SPACE 

CONTROL 

COMMUNICATIONS 

TRUST: 

TMD  SENSOR 

TECH.,  FIELD 
MEASUREMENTS,  & 
DATA  ANALYSIS 

DETECTION  AND 

I.D.  OF  L.O. 
AIRBORNE 
VEHICLES 

DETECTION  AND  I.D. 
OF  GROUND 
TARGETS 

DETECTION 
AND  I.D.  OF 
SPACE 
OBJECTS 

LONG-RANGE  A  J /. 
AND  LPI 

COMMUNICATIONS 

TO  MOBILE 
PLATFORMS 

FOCUS: 

INTEGRATED  TMD 
SYSTEM  TESTING, 
NMD  READINESS 

ADV.  SIGNAL 

PROC.,  ECCM 
TECHNIQUES, 
WIDE-AREA 
DEFENSES, 

SEEKER  ARCH. 

IMAGING  RADAR, 

ATR,  FOLIAGE- 
PENTRATiON 

GRD.-BASED 

RADAR,  GRD.- 

BASED/SPACE 

-BASED 

OPTICAUIR 

SENSORS 

ADVANCED  EHF 

COMM  SYSTEMS, 

HIGH-CAPACITY 

OPTICAL 

NETWORKS 

SOURCE:  Lincoln  Laboratory 
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The  Laboratory  reports  that  core  laboratory  work  is  defined  by  the  combination  of  the  Laboratory’s 
mission,  purpose,  and  capabilities,  by  Lincoln  Laboratory’s  core  competencies,  and  by  the  FFRDC 
special  relationship  maintained  between  the  Laboratory  and  the  DoD.  Core  work  has  the  following 
characteristics: 

Consistency  with  Lincoln  Laboratory’s  Mission ,  Purpose,  and  Capabilities.  The  work  assigned  to 
Lincoln  Laboratory  must  be  consistent  with  its  basic  mission  and  purpose,  must  be  sponsored  by  an 
appropriate  organization,  and  must  be  performed  effectively  using  the  capabilities  that  Lincoln 
Laboratory  maintains,  as  described  previously.  Lincoln  Laboratory  may  conduct  core  work  for  non- 
DoD  entities  when  it  is  directly  related  to  the  core  areas  defined  in  this  paper;  this  work  is  conducted 
subject  to  the  review  and  approval  process  described  in  Lincoln  Laboratory’s  Sponsoring  Agreement. 

Consistency  with  Lincoln  Laboratory’s  Core  Work.  The  work  assigned  to  Lincoln  Laboratory  must 
fall  within  Lincoln  Laboratory’s  core  work  as  described  previously,  including  research  and 
development  of  electronic  technologies. 

Consistency  with  Lincoln  Laboratory’s  Special  Relationship.  The  work  assigned  to  Lincoln 
Laboratory  must  be  appropriate  for  an  organization  maintaining  a  strategic  relationship  with  the  DoD, 
as  evidenced  by  the  need  for  one  or  more  of  the  following: 

•  Objective,  high-quality  work  on  subjects  integral  to  the  mission  of  the  sponsor 

•  Freedom  from  real  or  perceived  conflicts  of  interest 

•  Broad  access  to  information  of  a  sensitive  nature  to  the  sponsor 

•  Comprehensive  knowledge  of  sponsor  needs  and  problems 

•  Long-term  continuity  of  knowledge  of  sponsor  issues 

Work  is  accepted  that  conforms  to  the  core  requirements  and  is  within  the  scope  of  the  research  areas 
currently  being  pursued.  All  work  is  approved  by  the  Joint  Advisory  Committee.  According  to  the 
Sponsoring  Agreement,  “Any  agency  of  the  U.S.  Government  or  other  entity  included  Sections  5(1)  or 
(2),  may  request  the  laboratory  to  undertake  studies  or  development  programs  not  included  in  the 
program  originally  accepted  by  the  JAC.  Such  projects  will  require  approval  of  the  JAC  before  final 
acceptance  by  the  laboratory.” 

Non-DoD  sponsorship  currently  amounts  to  18  percent  of  staff  and  is  limited  by  policy  to  30 
percent. 

The  level  and  appropriateness  of  non-DoD  work  and  work  for  other  than  the  primary  sponsor. 

All  non-DoD  federally  sponsored  work  is  performed  on  the  Air  Force  contract  unless  specifically 
authorized  by  ESC/AX,  e  g.,  where  a  non-DoD  federal  sponsor  requests  a  separate  agreement.  Non- 
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DoD  federally  sponsored  work  must  be  consistent  with  the  DoD  Plan  for  Administration  of  Lincoln 
Laboratory  and  with  the  Sponsoring  Agreement. 

The  Sponsoring  Agreement  provides  that:  Support  to  non-U. S.  Government  organizations  may  be 
performed  under  separate  contract,  grant,  or  agreement.  According  to  the  agreement,  such  work  is 
limited  to  state  and  local  government  agencies,  educational  institutions  (including  MIT,  under 
interdepartmental  transfer  in  place  of  a  separate  contract,  grant,  or  agreement),  US.  allies/foreign 
governments,  and  non-profit  organizations.  Support  to  non-U.  S.  Government  organizations  may  be 
provided  only  where  the  scope  of  the  laboratory  support  represents  work  of  a  pre-competitive, 
collaborative,  R&D  nature  consistent  with  the  objectives  of  advancing  technology  and/or  transferring 
technology  developed  at  the  laboratory.  The  total  non-U.  S.  Government  funding  (including  CRADAs) 
may  not  exceed  30  percent  of  total  laboratory’s  funded  support  during  any  fiscal  year. 

The  Sponsoring  Agreement  also  covers  Cooperative  Research  and  Development  Agreements 
(CRADAs).  CRADAs  must  be  consistent  with  the  charter  and  mission  of  Lincoln  Laboratory.  ESC 
may  approve  CRADAs  provided  that  the  Lincoln  Laboratory  CRADA  partner  agrees  that  (a)  it  may 
not  submit  a  proposal  as  either  a  prime  contractor  or  a  subcontractor  for  any  future  acquisition  which 
identifies  that  Lincoln  Laboratory  or  Lincoln  Laboratory’s  technology  will  contribute  to  the 
performance  of  any  resulting  contract;  and  (b)  it  may  not  use  Lincoln  Laboratory’s  services  in  the 
performance  of  any  contract. 

The  Agreement  states  that  Lincoln  Laboratory  will  not  receive  money  from  partners  for  any  services 
rendered  in  areas  where  a  reasonable  perception  of  the  transaction  might  suggest  that  Lincoln’s 
provision  of  advice  to  its  government  sponsors  might  be  biased  thereby. 

Lincoln  Laboratory  also  participates  in  technology  transfer  activities  supporting  a  number  of 
government  programs.  The  Laboratory  submits  all  technology  transfer  proposals  to  the  contracting 
office  (ESC/AX)  for  review  and  approval. 

Compliance  with  the  DoD  Management  Plan 

The  DoD  plans  directs  Research  and  Development  (R&D)  Laboratories  to  maintain  expertise  and 
related  competencies  necessary  to  address  the  core  work  and  priorities  of  the  sponsor. 

More  generally,  the  Centers  are  directed  to  perform  only  core  work,  and  such  work  must  be  approved 
by  the  primary  sponsor;  can  only  be  accomplished  for  DoD,  other  government  entities  (including  state 
and  municipal  governments),  and  for  not-for-profit  activities.  It  cannot  include  commercial  work. 

Parent  institutions  operating  DoD  FFRDCs  may  perform  non-FFRDC  work,  but  such  work  must 
conform  to  some  specific  guidelines:  be  in  the  national  interest;  not  undermine  the  independence, 
objectivity,  and  credibility  of  the  institute  or  call  into  question  the  freedom  from  conflict  of  interest;  not 
take  unfair  advantage  of  FFRDC  work;  and  normally  work  in  the  public  sector,  although  commercial 
work  may  be  accepted  if  the  sponsor  agrees. 

Fees  may  be  authorized.  The  Plan  calls  for  “an  annual  fee  proposal  justifying  each  element  of  fee.” 
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Sponsors  are  directed  to  ensure  the  FFRDC  is  used  only  for  intended  purposes,  ensure  that  the  costs 
are  reasonable  and  products  are  of  high  value;  review  work  to  ensure  it  is  appropriate  to  be  done  and 
assure  DDR&E  that  this  is  all  being  done. 

Lincoln  Laboratory  states  that  it  does  maintain  a  capability  in  a  broad  range  of  technical  areas, 
consistent  with  DoD  needs.  It  specifies  six  specific  mission  areas  in  which  it  has  core  competencies. 
The  ability  to  continue  to  maintain  competencies  may  be  hampered  by  the  ceilings.  These  ceilings  have 
had  their  most  direct  impact  through  their  effects  on  the  necessary  purchase  of  experimental 
equipment. 

All  work  is  approved  by  the  sponsor  and  is  in  compliance  with  the  DoD  Management  Plan. 

There  is  no  fee. 

The  sponsor’s  management  to  ensure  product  and  cost  control  is  outlined  below. 

Sponsor’s  Management  Processes 

The  primary  sponsor  of  Lincoln  Laboratory  is  the  Air  Force.  The  administrative  guidance  and  support 
are  provided  by  a  Joint  Advisoiy  Committee  (JAC),  which  is  comprised  of  representatives  of  the  Air 
Force,  Army,  Navy,  ARP  A,  BMDO,  and  DDR&E.  The  composition  of  the  committee  is  shown  in 
table  1-3  below.  The  committee  is  responsible  for  what  work  is  accepted,  oversight  of  work,  funds, 
and  products. 


Table  1-3:  Joint  Advisory  Committee  Membership 

•  Director  of  Defense  Research  and  Engineering:  JAC  Chair 

•  Assistant  Secretary  of  the  Air  Force  (Acquisition) 

•  Assistant  Secretary  of  the  Army  (Research,  Development  and  Acquisition) 

•  Assistant  Secretary  of  Navy  (Research,  Development  and  Acquisition) 

•  Director,  ARPA 

•  Director,  BMDO 

•  JAC  Executive  Group  (EG)  Chair  (HQ  AFMC/ST) 

•  JAC  Administrative  Agent-Executive  Director,  Electronics  Systems  Center 


SOURCE:  U.S.  Air  Force 

The  JAC  meets  once  each  year  to  approve  the  program.  A  JAC  Executive  Group  (table  1-4)  also 
meets  once  a  year  and  helps  develop  the  program  for  the  JAC  approval. 
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Table  1-4:  Joint  Advisory  Committee  Executive  Group  Membership 


•  Chair  —  Director  of  Science  and  Technology,  Air  Force  Materiel  Command  (AFMC/ST) 

•  Vice  Chair  —  Executive  Director,  Electronic  Systems  Center  (ESC/CD) 

•  Navy  —  Chief  of  Naval  Research 

•  Army  --  Director  for  Space  &  Strategic  Technology,  Office  of  Assistant  Secretary  of  the  Army 
(RD&A) 

•  DARPA  —  Technical  Assistant  to  the  Director 

•  BMDO  —  Deputy  Director 

•  National  Security  Agency 

•  Air  Force  --  SMC/XR  (Space  and  Missile  Center  Plans) 

--  RL/CD  (Chief  Scientist  —  Rome  Laboratory) 

—  AFOSR/ST  (Director  of  Technology  Secretariat) 


SOURCE:  U.S.  Air  Force 

The  yearly  program  proposal  review  and  approval  process  is  shown  at  figure  1-5. 


Figure  1-5:  Yearly  Program  Proposal  Review  and  Approval  Process 


SOURCE:  Lincoln  Laboratory 

The  Sponsoring  Agreement  notes  that  all  DoD  work  is  to  be  performed  on  the  Air  Force  contract 
unless  specifically  authorized  by  ESC/AX.  On  approval  by  the  JAC,  the  Air  Force  Lincoln  Laboratory 
Project  Office  (ESC/AX)  prepares  a  Technical  Objectives  and  Plans  (TO&P)  document  or  modifies  an 
existing  TO&P  document  which  will  reflect  the  approved  program. 
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The  program  sponsor  is  expected  to  monitor  the  program  directly;  to  provide  technical  direction 
(through  appropriate  contractual  channels);  to  provide  sponsor  feedback  of  the  Lincoln  Laboratory 
efforts  annually  through  ESC/AX;  and  to  interact  frequently  with  the  laboratory  as  the  program  is 
carried  out.  Within  the  practical  limits  of  available  funds,  the  program  sponsor  agrees  to  provide  all 
necessary  support  for  the  program. 

The  program  oversight  process  is  shown  in  figure  1-6. 


Figure  1-6:  MIT  Lincoln  Laboratory  Program  Oversight 


SOURCE:  Lincoln  Laboratory 

All  sponsors  evaluate  the  work  done  for  them.  These  evaluations  are  consolidated  by  the  sponsor  and 
provided  as  feedback  to  Lincoln  Laboratory. 

To  provide  the  maximum  dissemination  of  technical  information  concerning  the  laboratory’s  programs, 
the  JAC  EG  sponsors  a  series  of  annual  technical  seminars  by  the  laboratory  covering  all  of  the 


1-11 


laboratory’s  major  programs  and  related  technology  efforts  grouped  by  appropriate  areas  (e.g.,  military 
satellites,  communications,  etc  ).  Attendance  at  these  seminars  is  by  invitation,  extended  by  the  JAC 
EG,  to  members  of  the  Executive  Group,  program  sponsors,  industry,  and  persons  from  other  DoD, 
other  government  and  FFRDC  organizations.  These  Seminars  are  designed  to  provide  peer  review  and 
validate  the  technical  direction  of  the  laboratoiy.  In  addition  to  disseminating  program  information 
through  the  technical  seminars,  the  laboratory  publishes  its  program  results  in  technical  reports, 
deposited  in  DTIC. 

To  provide  the  JAC  and  the  JAC  EG  with  descriptive  and  budget  material  to  assist  their  annual  reviews 
of  the  laboratory’s  programs,  Lincoln  Laboratory  prepares  annually  a  document  outlining  the 
laboratory’s  proposed  programs  and  associated  budgets  for  the  three  ensuing  fiscal  years.  This 
document  is  reviewed  by  program  sponsors  prior  to  submission  to  the  JAC  EG.  Further,  to  assist 
interaction  with  sponsors,  and  to  assist  the  JAC  EG,  and  the  JAC  with  long-range  planning  for  the 
laboratory’s  operations  and  programs,  the  laboratory  prepares  five-year  projections  of  its  major 
programs  or  areas  of  work.  The  five-year  projections  are  included  in  the  program  document  and  is 
updated  annually  in  preparation  for  the  JAC  EG  meeting.  This  program  will  be  presented  to  the  JAC 
EG  for  tentative  acceptance  and  establishment  of  a  financial  plan.  The  program,  with  revisions  by  the 
JAC  EG,  will  be  referred  to  Massachusetts  Institute  of  Technology  within  45  days  of  submission  for 
further  consideration  by  the  laboratory  who  will  prepare  a  program  consistent  with  the  financial  plan 
and  program  guidance  provided  by  the  JAC  EG.  This  revised  program  is  presented  to  the  JAC  for 
approval  not  later  than  August  for  the  ensuing  fiscal  year. 

The  Thoroughness  of  the  Five-Year  Review 

The  most  recent  Comprehensive  Review  of  the  MIT  Lincoln  Laboratory  is  dated  August  3, 1994.  The 
Review  addressed  the  topics  required  by  FAR  35.01 7:  technical  needs,  efficiency  and  effectiveness  in 
meeting  those  needs,  alternative  sources,  and  adequacy  of  the  FFRDC  management  in  ensuing  a  cost 
effective  operation. 

The  Review  discusses  the  continued  need  to  develop  new  defense  systems.  It  specifies  technology 
needs  in  the  area  of  solid  state  electronics,  radar  and  optical  sensors,  surgical  processing,  surveillance, 
military  satellite  communications,  spacecraft,  analog  and  digital  integrated  circuit  technology,  air  traffic 
control,  signal  intercept  technology,  high  energy  laser  beans  controls,  laser  devices,  optics,  antennas, 
electromagnetic  propagation,  and  strategic  and  tactical  systems  and  countermeasures.  The  Review 
states  that  these  needs  are  validated  in  the  extensive  interaction  with  the  DoD  user  community,  and 
examined  yearly  through  the  Laboratory  seminar  services,  and  oversight  process. 

The  Laboratory’s  effectiveness  and  efficiency  are  addressed  in  a  discussion  of  the  Annual  Sponsor 
Feedback  evaluation  that  is  required  from  each  government  sponsor.  The  results  of  the  process  are 
summarized  and  specific  problem  projects  are  discussed  in  more  detail  to  include  the  evaluations  of 
those  projects  and  the  Laboratory  response.  The  overwhelming  assessment  of  the  evaluation  process  is 
that  the  Laboratory  is  fulfilling  its  technical  responsibilities  in  an  outstanding  manner.  The  main 
criticisms  are  on  financial  reporting  with  the  projects. 
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The  assessment  of  alternative  sources  noted  that  such  sources  are  used  when  they  can  meet  the  long¬ 
term  research  and  development  requirements  of  the  sponsor  on  individual  tasks.  The  FFRDC  is  only 
used  when  other  sources  are  not  available.  The  actual  discussion  of  alternative  sources  is  very  general 
and  points  out  the  long-term  investments  that  the  government  has  made  in  the  Laboratory’s  physical 
plant  and  personnel  and  the  high  cost  to  replace  that  investment.  The  Sponsor  did  not  do  a  detailed 
assessment  of  the  costs  of  changing  the  provider,  but  noted  the  outstanding  capability  that  currently 
exists  and  the  absence  of  any  need  to  change  and  incur  these  new  costs.  Close  out  cost  alone  are  $236 
million. 

The  review  of  management  for  cost-effective  operations  reviews  the  responsibilities  for  financial 
oversight  and  the  review  systems  in  place. 
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ACQUISITION  AND 
TECHNOLOGY 


Appendix  J 

THE  UNDER  SECRETARY  OF  DEFENSE 
3010  DEFENSE  PENTAGON 
WASHINGTON,  D.C.  20301-3010 


OCT  3  0  1995 


memorandum  for  chairman,  defense  science  board 

SUBJECT:  Terms  of  Reference  —  Defense  Science  Board  FFRDC  and  UARC 

Independent  Advisory  Task  Force 

You  are  requested  to  establish  a  Defense  Science  Board  FFRDC  and 
UARC  Independen ^Advisory  Task  Force  to  review  and  provide  advice  on 

d“  s  S"ent  of  its  Federally  Funded  Research  and  Develo^nt 

Centers  (FFRDC)  and  University  Affiliated  Research  Center  (  - 

This  Task  Force  should  be  comprised  of  individuals  from  outside 
Government  who  will  help  establish  guidelines  and  conduct  a- 
independent  semi-annual  review  of  progress  against  recommendati  . 

The  Task  Force  should  review  and  advise  the  Department  on: 

the  ob-iectives  and  guidelines  for  appropriate  scope  of  work,  _ 

organizational  structure,  size,  appropriateness  of 

-  compliance  with  the  annual  DoD  Management  Plan  for  FFRDCs  and 

-  th^various  DoD  sponsor's  management  processes  for  FFRDCs  and 

the  level  and  appropriateness  of  non-DoD  work,  to  include  non 
sponsored  work  in  addition  to  non-DoD  work,  undertaken  y  o 

FFRDCs  and  UARCs;  .  , 

-  the  thoroughness  of  the  FFRDC  five  year  review  process;  and 

-  periodically  review  selected  FFRDC  and  UARC  programs. 

The  Task  Force  should  submit  its  first  report  by  March  31,  1996. 

The  Director,  Defense  Research  and  Engineering  will  5^“°' 

Task  Force  and  provide  support  as  may  be  necessary.  Mr.  Gordon  R. 
EnglandTwill11  serve  as  its  Chair.  Mr.  Charles  Kimzey  will  serve  as  the 
Task  Force  Executive  Secretary  and  CDR  Robert  C.  Hardee,  Y*  . 

“the  DSB  Secretariat  representative.  It  -^  anticipated 
thiq  Task  Force  will  need  to  go  into  any  particular  matters 
^hin  the  meaning  of  section  209  of  Title  18,  United  states  Code,  nor 
will  it  cause  any  member  to  be  placed  in  the  position  of  acting  as 
procurement  official. 


Paul  G .  Kaminski 


Findings  Relationship  to  Terms  of  Reference  Findings 

•  Objectives  and  guidelines  for  scope  of  work,  organizational  structure,  size,  and 
appropriateness  of  customers 


—  Strategic  Relationship . p.  15 

--  Competition . p.  16 

—  Core  Competencies . p.  17 

—  Effects  of  Current  Structure . p.  1 8 

--  Funding . p.  20 


•  Compliance  with  the  DoD  Management  Plan 


—  Some  Positive  Trends . p.  14 

—  Core  Competencies . p.  17 

—  The  Burdens  of  Current  DoD  Policy . p.  18 

—  Management  Plan . p.  19 

--  Allocation  Process . p.  21 

—  Excessive  Oversight . p.  2 1 

--  Costs . p.  22 

--  Acknowledge  Differences . p.  23 


•  Sponsors  ’  management  process 


—  Core  Competencies . p.  17 

--  Five-Year  Reviews . p.  19 

--  Funding . p.  20 

—  Excessive  Oversight . p.21 

--  Independent  Research  and  Development . p.  21 

--  OSD  Staffing . p.  21 

--  Congressional  Ceiling . p.  22 


•  The  level  and  appropriateness  of  non-DoD  work  and  work  for  other  than  the  primary 
sponsor 

--  Other  Agency  Tasking . p.  23 

•  The  Thoroughness  of  the  Five-Year  Reviews 


—  Strategic  Relationship . p.  15 

--  Core  Competencies . p.  17 

—  Five-Y ear  Reviews . p.  19 

—  Costs . p.  22 
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Appendix  K 


Defense  Science  Board  FFRDC  &  UARC 
Independent  Advisory  Task  Force 

Mr.  Gordon  R.  England,  Chairman 
GRE  Consultants,  Inc. 

4408  Lost  Creek  Boulevard 
Aledo,  TX  76008 
TEL:  817/560-2203 
FAX:  817/560-2298 


Mr.  G.  Dean  Clubb,  President 
Systems  Group 
Texas  Instruments,  Inc. 

P.O.  Box  401,  M/S-3425 
Lewisville,  TX  75067 
TEL:  214/462-6400 
FAX:  214/462-6423 
E-mail:  gdc@mag.ti.com 

Mr.  Robert  B.  Costello,  Consultant 

77  Tidy  Island 

Bradenton,  FL  34210 

TEL:  813/794-6642 

FAX:  813/794-6642 

(call  first) 

Mr.  Rhett  B.  Dawson,  President 
Information  Technology  Industry 
Council 

1250  Eye  Street,  N.W.,  Suite  200 
Washington,  DC  20005 
TEL:  202/626-5757 
202/737-8888 
FAX:  202/638-4922 

Mr.  Arthur  E.  Johnson 
Group  Vice  President 
Loral  Federal  Systems  Group 
8201  Greensboro  Drive 
McLean,  VA  22102 
TEL:  703/287-3300 
FAX:  703/287-3340 


Mr.  Herman  E.  Shipley,  Consultant 
3800  N.  Fairfax  Drive,  Apt.  1809 
Arlington,  VA  22203-1706 
TEL:  703/527-7270 
FAX:  703/276-2620 

Government  Representatives 

Mr.  Charles  Kimzey,  Executive  Secretary 
5203  Leesburg  Pike,  Suite  1403 
Skyline  2 

Falls  Church,  VA  22041 
TEL:  703/681-5453 
FAX:  703/681-3722 
E-mail :  kimzeych@acq .  osd .  mil 

CDR  Robert  C.  Hardee,  USN 
DSB  Secretariat  Representative 
Defense  Science  Board,  OUSD  (A&T) 

3 140  Defense  Pentagon 
Washington,  DC  20301-3140 
TEL:  703/695-4157 
FAX:  703/697-1860 
E-mail:  hardeerc@acq.osd.mil 


Appendix  L:  Meeting  Agendas 
AGENDA 

DEFENSE  SCIENCE  BOARD 
FFRDC/UARC  Independent  Advisory  Task  Force 

U.S.  Air  Force  Association 
1501  Lee  Highway 
4th  Floor  Conference  Center 
Arlington,  VA  22204 
703/247-5800 

December  12-13, 1995 


December  12, 1995 


8:30 

Welcome 

General  John  Shaud,  USAF  (Ret) 
Executive  Director 

U.S.  Air  Force  Association 

8:35 

Task  Force  Introductions 

Mr.  Gordon  England 

Task  Force  Chairman 

8:45 

Task  Force  Mission 

The  Honorable  Anita  Jones,  Director 
Defense,  Research  &  Engineering 

9:30 

Government  Conflict  of 

Interest  Requirements 

Mr.  Calvin  Vos,  Senior  Attorney 
Standards  of  Conduct  Office 

Office  of  the  General  Counsel 

9:40 

DSB  FFRDC  Task  Force  (April  1995) 

Mr.  A.  Thomas  Young 

Former  President,  Martin  Marietta; 
Retired  Executive  Vice  President, 
Lockheed  Martin 

11:00 

Committee  Discussion 

Mr.  England 

12:00 

Lunch 

1:00 

DoD  FFRDC  Management  Plan 

Mr.  Robert  Nemetz,  Director 

Studies  and  FFRDC  Programs 

Office  of  the  Secretary  of  Defense 

2:30 

Break 

2:45 

Committee  Discussion 

Mr.  England 

4:00 

Industry  Perspectives  on 

Mr.  Bert  Concklin,  President 

FFRDC  Management 

Professional  Services  Council 

5:00 

Adjourn 

December  13, 1995 

8:30 

Sponsor’s  Management  Process 

Dr.  Donald  Daniel 

Deputy  Director 

Science  and  Technology  Directorate 
U.S.  Air  Force  Materiel  Command 
MIT  Lincoln  Laboratory  Sponsor 

9:15  FFRDC  Management  Process  Mr.  Donald  MacLellan 

Assistant  Director 
MIT  Lincoln  Laboratory 

10:00  Break 

10:15  Sponsor’s  Management  Process  Capt.  W.S.  Slocum,  Deputy  Director 

Assessment  Division,  N81B 
Center  for  Naval  Analyses  Sponsor 

11:00  FFRDC  Management  Process  Dr.  Robert  J.  Murray 

President 

Center  for  Naval  Analyses 

11:45  Committee  Discussion  Mr.  England 

12:30  Adjourn 
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AGENDA 


DEFENSE  SCIENCE  BOARD 
FFRDC/UARC  Independent  Advisory  Task  Force 

THE  AEROSPACE  CORPORATION 
2350  East  El  Segundo  Blvd. 

El  Segundo,  CA 

JANUARY  31, 1996 


8:00 

Task  Force  Assembles 

Task  Force 

Aerospace  Executive  Council 
SMC  Senior  Leadership 

8:15 

Task  Force  Introductions  and 

Terms  of  Reference 

Mr.  Gordon  England 

Task  Force  Chairman 

8:20 

UNCLASSIFIED  Aerospace  and  SMC 
briefings  and  discussions 

Aerospace  (2  hours) 

SMC  (1  hour) 

Mr.  E.C.  (Pete)  Aldridge,  Jr. 
President  and  CEO 

Lt.  Gen.  Lester  L.  Lyles 
Commander,  SMC 

10:30 

Break 

11:30 

Public  attendees  depart  for  lunch 

11:45 

Task  Force,  Executive  Council  and 
guests  depart  for  lunch 

1:00 

Classified  Portion  of  the  visit. 

Facilities  Tour  and  other  Discussions 

Task  Force 

Mr.  Aldridge 

2:45 

Closing  discussions 

Mr.  Gordon  England 

Task  Force  Chairman 

3:30 

Task  Force  departs 
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AGENDA 


DEFENSE  SCIENCE  BOARD 
FFRDC/UARC  Independent  Advisory  Task  Force 

THE  RAND  CORPORATION 
1700  Main  Street 
Santa  Monica,  CA 

FEBRUARY  1, 1996 


8:00 

Introductions  and  Opening  Comments 

Gordon  England 

Task  Force  Chairman 

8:15 

Introductory  Overview  of  RAND 

James  Thomson,  President  and 
Chief  Executive  Officer 

Michael  Rich,  Executive 

Vice  President 

8:45 

National  Defense  Research  Institute  (NDRI) 
Sponsor’s  Management  Process 

Robert  Soule 

Principal  Deputy  Director,  PA&E, 
OSD 

9:15 

NDRI’s  Management  Process 

David  Gompert,  Vice  President 
and  Director,  NRDI 

10:15 

Break 

10:30 

Arroyo  Center 

Sponsor’s  Management  Process 

Maj.  Gen.  David  Heebner,  Director 
Program  Analysis  and  Evaluation 
U.S.  Army  Staff 

11:00 

12:00 

Arroyo  Center’s  Management  Process 

Break  to  prepare  for  working  lunch 

James  Quinlivan,  Vice  President 
and  Director,  Arroyo  Center 

12:15 

Project  AIR  FORCE 

Sponsor’s  Management  Process 

Maj.  Gen.  Robert  Linhard 

Director  of  Plans 

U.S.  Air  Force  Staff 

12:45 

Project  AIR  FORCE’S  Management  Process 

Brent  Bradley,  Vice  President 
and  Director,  PAF 

1:45 

Break  //  Begin  Closed  Session 
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2:00 

Cross-cutting  Issues 

2:45 

Wrap-up 

3:00 

Facility  Tour 

4:00 

Adjourn 

James  Thomson,  President  and  CEO 
Michael  Rich 
Executive  Vice  President 

Gordon  England 
Task  Force  Chairman 
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AGENDA 


DSB  TASK  FORCE  VISIT  TO  MITRE 
THE  MITRE  CORPORATION 
202  Burlington  Road 
Bedford,  MA  01730 
“A”  Building 

March  1, 1996 


8:00  Introductions 


Mr.  Gordon  England 
Task  Force  Chairman 


8:10 

9:15 

9:45 

10:15 

10:30 

11:00 

11:30 

12:15 

1:15 

1:30 

2:00 


4:00 


Welcome  and  MITRE  Overview  of  Corporate  History,  Mr.  Vic  DeMarines 

Split,  Structure,  Sponsors  MITRE  President/CEO 


Primary  Sponsor  Briefing  Mr.  Tony  Valletta 

Deputy  ASC  (C3I) 

MITRE  Briefing  on  C3I  Challenge  and  Response  Mr.  Vic  DeMarines 

Break 


Army  Briefing  on  Army  use  of  MITRE  and 
Management  of  DoD  Work 


Air  Force  Briefing  on  Air  Force  Use  of  MITRE 
and  Management  of  Air  Force  Work 

MITRE  Briefings  on  C3I FFRDC  Core  Work 
and  MITRE  Operations,  Business  and  Financial 
Practices  and  Metrics 

(Break  to  pick  up  box  lunch  in  meeting  area) 
MITRE  Summary 
Discussion  and  Wrap  Up 


Mr.  Vic  Ferlise 
Deputy  to  Commanding 
General,  Army  CECOM 

Lt.  Gen.  Franklin 
Commander,  Air  Force  ESC 

Mr.  Vic  DeMarines,  et  al. 


Mr.  Vic  DeMarines 
Mr.  Gordon  England 


Tour  of  MITRE  Facility.  For  Task  Force  members.  Government  officials, 
and  MITRE  escorts  only.  Demonstrations  of  several  MITRE  projects  that 
integrate  new  COTS  (commercial  off-the-shelf)  capabilities  with  existing 
DoD  systems. 


Adjourn 
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AGENDA 


DEFENSE  SCIENCE  BOARD 
FFRDC/UARC  IndependentAdvisory  Task  Force 

Room  123,  Building  2001 
Institute  for  Defense  Analysis 
1801  North  Beauregard  Street 
Alexandria,  VA  22311-1772 

April  9, 1996 


8:45 

Introductions 

9:00 

IDA  Organization  and 

Management  Briefing 

9:45 

IDA  Sponsor  Briefing 

Mr.  Gordon  England 
Task  Force  Chairman 

General  Larry  D.  Welch 
(USAFRet.) 

President,  IDA 

Mr.  Irving  Blickstein 
Director,  Acquisition 
Program  Integration, 

OSD 

Hon.  Philip  Coyle,  Director 
Operational  Test  and 
Evaluation,  OSD 


10:30  Break 

10:45  LMI  Sponsor  Briefing 


11:15  LMI  Organization  and 

Management  Briefing 


Mr.  Roy  Willis 
Principal  Deputy  Under 
Secretary  of  Defense 
(Logistics) 

General  William  G.T.  Tuttle 
(USA  Ret.) 

President,  LMI 


12:00  Lunch 
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12:35 

2:00 

2:30 

3:00 

5:00 


DoD  Management  Briefing 


SEI  Sponsor  Briefing 


SEI  Organization  and 
Management  Briefing 

Panel  Discussion  Strategic 
Relationships,  Access  to  Proprietary 
Data  and  Conflict  of  Interest 


Mr.  Robert  Nemetz 
Director,  OSD  Studies  and 
FFRDC  Programs 

Mr.  Robert  J.  Kent 
Director,  ESC 
Software  Center 

Dr.  Larry  Druffel 
Director,  SEI 

Mr.  John  Stewart 
McKinsey  &  Company; 
Mr.  William  Harmon 
Perot  Systems 


Adjourn 
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AGENDA 


DEFENSE  SCIENCE  BOARD 
FFRDC/UARC  Independent  Advisory  Task  Force 

Center  for  Naval  Analyses 
4401  Ford  Avenue 
Board  Room 
Alexandria,  VA 

May  22, 1996 


8:00 

Welcome 

Dr.  Robert  Murray 

President,  CNA 

8:15 

Introduction  and  Agenda 

Gordon  England 

Chairman 

8:30 

DoD  Inspector  General 

Garold  E.  Stephenson 

FFRDC  Audits 

Audit  Program  Director 

9:30 

General  Accounting  Office 

Charles  W.  Thompson 

FFRDC  Audits 

Asst.  Dir.,  Defense  Acq.  Issues 
Nat’l  Sec’y  Int’l  Affairs  Div. 

10:30 

Break 

10:45 

DoD  FFRDC  Management  Plan 

Robert  Nemetz 

Director,  OSD  Studies  and 
FFRDC  Program 

11:45 

RAND 

James  Thomson 

President  &  CEO,  RAND 

12:30 

Lunch 

All 

1:15 

Center  for  Naval  Analyses 

Dr.  Robert  Murray 

President,  CNA 

2:00 

Logistics  Management  Institute 

Gen.  William  Tuttle 

President,  LMI 

2:45 

Break 
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3:00 


MITRE 


Victor  DeMarines 
President  &  CEO,  MITRE 


3:45  Discussion 


4:30  Adjourn 


Gordon  England 
Chairman 
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Appendix  M 


Report  of  the  Defense  Science  Board  Task  Force  on  the  Role  of  Federally  Funded 
Research  and  Development  Centers  in  the  Mission  of  the  Department  of  Defense 

The  Task  Force  was  established  in  1994  and  reported  its  findings  in  April  1995.  One  of  its 
key  findings  was  that  there  is  a  lack  of  trust  on  how  the  DoD  uses  its  FFRDC’ s.  This 
arises  from  a  lack  of  transparency  in  the  Departments’  FFRDC  management  process.  To 
address  this  trust  issue,  and  still  retain  the  highly  valued  intimacy  of  FFRDCs  with  the 
government,  the  Task  Force  reported  that  it  believed  that  new  management  tools  needed 
to  be  put  in  place  to  provide  public  visibility  into  the  DoD  decision  processes.  The  Task 
Force  recommended  that,  at  a  minimum,  an  independent  review  panel  involving  highly 
respected  personnel  from  outside  of  DoD  be  established  to  systematically  review  and 
advise  the  Department  on  its  management,  use,  and  oversight  of  its  FFRDCs.  The  DSB 
FFRDC/UARC  Independent  Advisory  was  established  as  a  result  of  this  recommendation. 

The  Report  of  the  Task  Force  contains  a  number  of  other  findings  that  are  important  to 
the  conduct  of  the  current  Task  Force.  These  findings  are  summarized  below. 

First,  the  Task  Force  concluded  that  DoD  continues  to  have  a  need  for  FFRDC -like 
organizations  who  have  special  relationships  with  the  Department  and  include  the 
following  characteristics: 

•  have  unique  competence  and  quality, 

•  are  closely  integrated  with  their  sponsor, 

•  adhere  to  strict  constraints  to  minimize  conflicts  of  interest,  to  promote  objectivity, 
and  to  ensure  independence  from  interests  that  may  conflict  with  sponsor  interests,  and 

•  maintain  continuity  of  relationship  sufficient  to  establish  “corporate  memory5  in  topics 
of  critical  interest  to  the  sponsor. 

Second,  the  Task  Force  reported  that  the  set  of  organizations  now  designated  as  DoD 
FFRDCs  are  judged  to  be  of  high  quality,  as  viewed  by  their  sponsors,  other  private  sector 
organizations,  and  the  Task  Force.  However,  the  congressionally  mandated  salary 
constraints  that  are  unique  to  FFRDCs  were  viewed  as  inappropriate  and  the  Task  Force 
wrote  that  the  constraints  will  lead  to  a  significant  degradation  in  the  competence  and 
quality  of  the  individuals  and  the  organizations. 

Third,  the  Task  Force  concluded  that  there  is  a  “core”  work  effort  at  each  FFRDC  which 
requires  the  special  relationships  described  above.  The  T ask  Force  stated  that  it  did  not 
have  confidence  that  this  work  could  be  offered  for  full  and  open  competition  without 
losing  some  of  the  essence  and  value  of  the  service.  The  Task  Force  noted,  however,  that 


other,  “non-core”  efforts  at  some  of  the  FFRDCs  do  not  demand  the  special  FFRDC 
relationship  and  should  be  offered  for  open  commercial  competition  to  the  private  sector. 

Fifth,  the  Task  Force  noted  the  problem  of  trust  mentioned  earlier.  This  problem  arises 
from  the  DoD  use  of  its  FFRDCs  that  has  translated  into  a  serious  management  and 
political  issue.  Even  though  there  appeared  to  be  a  high  degree  of  trust  in  the  quality  of 
the  FFRDCs  work,  a  significant  distrust  of  DoD’ s  management,  use,  and  oversight  of  its 
FFRDCs  was  noted.  The  Task  Force  noted  a  specific  concern  that  the  FFRDCs  might 
gain  unfair  advantage  through  their  special  relationship  which  results  in  work  being 
sourced  to  them  that  could  as  effectively  and  appropriately  have  been  openly  competed 
for.  The  Task  Force  reported  that  the  internal  government  decision  process  that  allocates 
this  work  is  not  accessible  for  review  by  those  who  feel  the  inequity  of  the  outcome.  The 
Task  Force  recommended  an  independent  review  to  advise  DoD  on  its  management  and 
oversight  of  the  FFRDC. 

Sixth,  the  Task  Force  concluded  that  all  the  FFRDCs  should  be  retained,  but  the  Task 
Force  concluded  that  the  diversification  of  either  FFRDCs  or  their  parent  corporations 
into  areas  of  work  outside  their  core  domain  should  be  limited  to  that  which  DoD  judges 
to  be  in  the  best  interest  of  the  country. 

Finally,  the  Task  Force  argued  that  this  solution  will  not  be  effective  so  long  as  an  artificial 
cap,  which  does  not  represent  the  natural  need  of  the  Department  for  these  capabilities  and 
the  ability  of  the  FFRDCs  to  serve  that  need,  exists  on  the  FFRDC  work. 

The  Task  Force  outlined  three  basic  options  available  to  the  DoD  for  improving  its 
acquisition  of  FFRDC-like  services: 

•  Option  1  is  to  work  harder  at  preserving  the  original  FFRDC  concept.  The  Task 
Force  stated  that  under  this  option,  the  DoD  would  define  and  refine  the  scope  of 
work  to  assure  that  it  fits  the  “core”  criteria  of  work  for  the  FFRDCs,  redouble  its 
internal  effort  to  limit  sole  sourcing  to  the  FFRDCs  to  that  criteria,  and  augment  the 
internal  process  with  the  independent  review  panel.  Diversification  of  effort  by  an 
FFRDC  or  its  parent  to  work  outside  of  its  core  domain  would  be  approved  only  if  in 
the  best  interests  of  the  country.  When  these  conditions  were  in  place,  the 
congressionally  mandated  ceilings  would  be  lifted  to  allow  for  a  level  of  effort 
corresponding  to  the  needs  of  the  government  and  the  capabilities  of  the  FFRDCs. 

•  Option  2  would  allow  the  non-profit  parent  of  an  FFRDC  activity  to  contain  a  clearly 
separated  non-FFRDC  segment  that  would  be  free  to  compete  for  work  outside  of 
the  FFRDC  domain,  so  long  as  the  effort  did  not  violate  the  conflict  of  interest 
requirements  of  the  FFRDC  activity. 

•  Option  3  would  terminate  the  use  of  the  FFRDC  designation  in  the  DoD  and  make 
all  of  this  work  eligible  for  competition.  Long  term,  intimate  relationships  would  still 
be  sought  with  private  sector  organizations,  but  the  mechanism  for  achieving  such 
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relationships  would  be  the  terms  and  conditions  of  long-term  (up  to  15-year) 
contracts,  rather  than  the  conditions  defined  by  the  FAR  for  FFRDCs. 

The  DSB  Task  Force  on  the  Role  of  FFRDCs  unanimously  recommended  Option  1  for  the 
Laboratory  and  Studies  and  Analyses  FFRDCs.  The  Task  Force  concluded  that  the 
argument  for  special  relationships  is  strong,  the  risk  of  the  competitive  process  for  their 
functions  high,  and  the  controversial  work  content  of  these  FFRDCs  low. 

The  conclusions  on  The  Aerospace  Corporation  and  MITRE,  however  were  more 
difficult.  Here  the  Task  Force  noted  that  the  nature  of  the  work  is  similar  to  that  which 
many  for-profit  companies  are  capable  of  doing  and  the  amount  of  work  is  substantial. 

The  Task  Force  reported  that  after  extensive  debate,  it  achieved  consensus  on  the 
desirability  of  Option  1  for  Aerospace  and  MITRE  as  well.  The  value  of  the  special 
relationship  was  judged  to  be  a  paramount  factor.  The  Task  Force  members  concluded 
that  both  Option  2  and  3  would  risk  the  loss  of  those  relationships. 


Appendix  N:  OFPP  Policy  Letter  84-1  and  Federal  Acquisition  Regulation  35.017 

EXECUTIVE  OFFICE  OF  THE  PRESIDENT 
OFFICE  OF  MANAGEMENT  AND  BUDGET 

WASHINGTON.  D.C.  20503 


OFFICE  OF  FEDERAL 

PROCUREMENT 

POLICY 


APR  4  1984 


OFPP  POLICY  LETTER  84-1 

TO  THE  HEADS  OF  EXECUTIVE  DEPARTMENTS  AND  ESTABLISHMENTS 
SUB3ECT:  Federally  Funded  Research  and  Development  Centers 


1  Purpose.  This  policy  letter  establishes  Government-wide  policies  for  the 
U&  use,  periodic  review,  end  termination  of  the  sponsorth.p  of 
Federally  Funded  Research  and  Development  Centers  (FFRDCs;. 

2.  Supersession.  Memorandum  from  the  Chairman  to  the  Members  of  the  Federal 
Council  for*"ScIence  and  Technology,  dated  November  1,  1967,  which  set  forth 
criteria  for  identification  of  FFRDCs  and  the  requirement  for  a  master 
Government  listing  of  these  centers,  is  superseded  by  this  policy  letter. 

3.  Authority.  This  policy  letter  is  being  issued  pursuant  to  Section^  6(a),  6W>U) 
and  6(d)(8)  of  the  Office  of  Federal  Procurement  Policy  Act,  as  amended,  41 
U.S.C.  405  (a),  (d)(1)  and  (d)(8),  which  empower  the  Administrator  of  OFPP  to 
prescribe  Government-wide  procurement  policies  and  to  complete  action  on  the 
recommendations  of  the  Commission  on  Government  Procurement. 

4.  Background.  The  Departments  of  Energy,  Defense,  Health  and^  Human 
Services,  the  National  Aeronautics  and  Space  Administration  and  the  National 
i^Uce  ’Foundation  currently  sponsor  a  total  of  34  FFRDCs.  Non-sponsonng 
departments  and  agencies  also  utilize  these  FFRDCs.  Federal  funding  of  FFRDC  s 
currently  exceeds  4  billion  dollars  per  year* 

In  1967,  a  Government-wide  policy  for  the  identification  and  maintenance  of  a 
master  listing  of  these  FFRDCs  was  issued  (reference  paragraph  2  -  Supersession). 
In  1972,  the  Commission  on  Government  Procurement  recommended  that  the 
Federal  Government  keep  open  the  option  to  organize  and  use  FFRDCs  to  sat  y 
needs  that  cannot  be  satisfied  effectively  by  other  organizational  resources.  The 
Commission  also  recommended  that  agency  heads  periodically  review  the 
continuing  need  for  existing  FFRDCs  and  approve  any  proposal  for  new  FFRDCs, 
with  specific  attention  paid  to  the  method  of  ultimate  termination  of  sponsorship. 
This  policy  letter  is  based  on  the  executive  branch  consideration  of  the 
Commission's  recommendations. 


5.  Definitions 

a.  Primary  Sponsor  —  The  executive  agency  which  manages,  administers  or 
monitors  overall  use  of  the  FFRDC. 
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b.  Sponsor  means  an  executive  agency  which  funds  and  monitors  specific 
work  of  a  continuing  nature  with  an  FFRDC  and  is  party  to  a  sponsoring 
agreement.  Multiple  sponsorship  of  an  FFRDC  is  possible  so  long  as  one  agency 
agrees  to  act  as  the  primary  sponsor  for  administrative  purposes. 

c.  Federally  Funded  Research  and  Development  Center  (FFRDC). 

(1)  FFRDCs  do  not  have  a  prescribed  organizational  structure.  They  can 
range  from  the  traditional  contractor-owned/contractor-operated  or  Gov¬ 
ernment-owned/contractor-operated  (COCO)  organizational  structures  to 
various  degrees  of  Contractor/Government  control  and  ownership.  In 
general,  however,  all  of  the  following  criteria  should  be  met  before  an 
activity  is  identified  .as  an  FFRDC: 

(a)  Performs,  analyzes,  integrates,  supports  (non-financial)  and/or 
manages  basic  research,  applied  research,  and/or  development. 
(Activities  primarily  engaged  in  routine  quality  control  and 
testing,  routine  service  activities,  production,  mapping  and 
surveys,  and  information  dissemination,  even  though  otherwise 
meeting  the  requirements  of  paragraph  5.C.,  are  specifically 
excluded  from  FFRDC  designation). 

(b)  Performance  of  the  functions  in  5.c.(l)(a)  is  either  upon  the 
direct  request  of  the  Government  or  under  a  broad  charter  from 
the  Government,  but  in  either  case  the  results  are  directly 
monitored  by  the  Government.  However,  the  monitoring  shall 
not  be  such  as  to  create  a  personal  services  relationship,  or  to 
cause  disruptions  that  are  detrimental  to  the  productivity 
and/or  quality  of  the  FFRDCs  work. 

^  majority  of  the  activity's  financial  support  (70%  or  more)  is 

received  from  the  Government  with  a  single  agency  usually 
predominating  in  that  financial  support. 

(d)  In  general,  most  or  all  of  the  facilities  are  owned  by  the 
Government  or  funded,  under  contract,  by  the  Government. 

^  *^he  activity  is  operated,  managed  and/or  administered  by 
either  a  university  or  consortium  of  universities,  other  non¬ 
profit.  organization  or  industrial  firm  as  an  autonomous 
organization  or  as  an  identifiable  separate  operating  unit  of  a 
parent  organization. 

(f)  A  long  term  relationship  evidenced  by  specific  agreement  exists 
or  is  expected  to  exist  between  the  operator,  manager,  or 
administrator  of  the  activity  and  its  primary  sponsor. 
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(2)  In  addition  to  the  above  criteria,  the  relationship  between  the 

activity  and  the  Government  should  exhibit  the  following 

characteristics  in  order  to  qualify  for  FFRDC  identification: 

(a)  The  activity  (organization  and/or  facilities)  is  brought  into 
existence  at  the  initiative  of  a  Government  agency  or  bureau  to 
meet  some  special  research  or  development  need  which,  at  the 
time,  cannot  be  met  as  effectively  by  existing  in-house  or 
contractor  resources. 

(b)  Work  from  other  than  a  sponsoring  agency  is  undertaken  only  to 
the  extent  permitted  by  the  sponsoring  agency  and  in 
accordance  with  the  procedures  of  the  sponsoring  agency. 

c)  The  activity,  whether  the  operator  of  its  own  or  a  Government- 
owned  facility,  has  access,  beyond  that  which  is  common  to  the 
normal  contractual  relationship,  to  Government  and/or  supplier 
data,  employees,  and  facilities  needed  to  discharge  its 
responsibilities  efficiently  and  effectively,  whether  the  data  is 
sensitive/proprietary  or  not. 

d)  The  primary  sponsor  undertakes  the  responsibility  to  assure  a 
reasonable  continuity  in  the  level  of  support  to  the  activity 
consistent  with  the  agency's  need  for  the  activity  and  the  terms 
of  the  sponsoring  agreement. 

e)  The  activity  is  required  to  conduct  its  business  in  a  responsible 
manner  befitting  its  special  relationship  with  the  Government, 
to  operate  in  the  public  interest  free  from  organizational 
conflict  of  interest,  and  to  disclose  its  affairs  (as  an  FFRDC)  to 
the  primary  sponsor. 

6.  Policy. 

a.  General.  Agencies  will  rely,  to  the  extent  practicable,  on  existing  in- 
house  and  contractor  sources  for  satisfying  their  special  research  or  development 
needs  consistent  with  established  procedures  under  The  Economy  Act  of  1932  (31 
USC  1535),  other  statutory  authority  or  procurement/assistance  regulations.  A 
thorough  assessment  of  existing  alternative  sources  for  meeting  these  needs  is 
especially  important  prior  to  establishing  an  FF^-DC.  This  Policy  Letter  does  not 
apply  to  the  performance  of  commercial  activities.  Performance  of  commercial 
activities  is  governed  by  OMB  Circular  No.  A-76. 
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b.  Establishment  of  an  FFRDC.  In  establishing  an  FFRDC,  the  sponsoring 
agency  shall  ensure  that: 

(1)  Existing  alternative  sources  for  satisfying  agency  requirements 
cannot  effectively  meet  the  special  research  or  development  needs 
(6.a). 

(2)  At  least  three  notices  are  placed  over  a  90-day  period  in  the 
Commerce  Business  Daily  and  The  Federal  Reeister  indicating  the 
agency's  intention  to  sponsor  an  FFRDC  and  the  scope  and  nature  of 
the  effort  to  be  performed  by  the  FFRDC. 

(3)  There  is  sufficient  Government  expertise  available  to  adequately  and 
objectively  evaluate  the  work  to  be  performed  by  the  FFRDC. 

(4)  Controls  are  established  to  ensure  that  the  costs  of  the  services  being 
provided  to  the  Government  are  reasonable. 

(5)  The  responsibility  for  capitalization  of  the  FFRDC  has  been  defined 
in  such  a  manner  that  ownership  of  assets  may  be  readily  and 
equitably  determined  upon  termination  of  the  FFRDC  relationship 
with  its  sponsor(s). 

(6)  The  purpose,  mission  and  general  scope  of  effort  of  the  FFRDC  is 
stated  clearly  enough  to  enable  differentiation  between  work  which 
should  be  performed  by  the  FFRDC  and  that  which  should  be 
performed  by  a  non-FFRDC. 

c.  Sponsoring  Agreements.  When  FFRDCs  are  established,  long-term 
Government  relationships  are  encouraged  in  order  to  provide  the  continuity  that 
will  attract  high  quality  personnel  to  the  FFRDC.  This  relationship  should  be  of  a 
•type  to  encourage  the  FFRDC  to  maintain  currency  in  its  field(s)  of  expertise, 
maintain  its  objectivity  and  independence,  preserve  its  familiarity  with  the  needs 
of  its  sponsor(s),  and  provide  a  quick  response  capability.  A  contract  is  the 
generally  preferred  instrument  under  which  an  FFRDC  accomplishes  effort  for  its 
sponsor(s).  However,  there  may  be  instances  where  other  legal  instruments  may  be 
appropriate.  A  written  agreement  of  sponsorship  between  the  FFRDC  and  its 
sponsor  or  primary  sponsor  where  more  than  one  sponsor  is  involved  may  be  used  in 
addition  to  the  contract  or  other  legal  instrument  under  which  an  FFRDC 
accomplishes  effort.  The  specific  content  of  a  sponsoring  agreement  will  vary 
depending  on  the  situation.  However,  there  are  certain  areas  common  to  ail 
situations  that  must  be  addressed.  The  following  requirements  must  be  addressed 
in  either  a  contract,  a  sponsoring  agreement  or  sponsoring  agency's  policies  and 
procedures. 
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(1)  Mandatory  Requirements 

(a)  A  delineation  of  the  purpose  for  which  the  FFRDC  is  being  brought  into 
being  -along  with  a  description  of  its  mission,  general  scope  of  effort 
envisioned  to  be  performed,  and  the  role  the  FFRDC  is  to  have  in 
accomplishment  of  the  sponsoring  agency's  mission.  This  delineation  must 
be  consistent  with  the  definition  of  an  FFRDC  set  forth  in  paragraph 
5.c(l)(a)  and  will  be  sufficiently  descriptive  so  that  work  to  be  performed 
by  the  FFRDC  can  be  determined  to  be  within  the  purpose,  mission  and 
general  scope  of  effort  for  which  the  FFRDC  was  established  and 
differentiated  from  work  which  should  be  performed  by  a  non-FrRDC. 
This  delineation  shall  constitute  the  base  against  which  changes  in  an 
existing  FFRDC's  purpose,  mission  or  general  scope  of  effort  will  be 
measured. 

(b)  Provisions  for  the  orderly  termination  or  nonrenewal  of  the  agreement, 
disposal  of  assets  and  settlement  of  liabilities.  The  term  of  the 
sponsorsing  agreement  will  not  exceed  five  years  but  can  be  renewed,  as  a 
result  of  periodic  review,  in  not  to  exceed  five  year  increments. 

(c)  A  prohibition  against  the  FFRDC  competing  with  any  non-FFRDC  concern 
in  response  to  a  Federal  agency  formal  Request  For  Proposal  for  other 
than  the  operation  of  an  FFRDC.  This  prohibition  is  not  required  to  be 
applied  to  any  parent  organization  or  other  subsidiary  of  the  parent 
organization  in  its  non-FFRDC  operations.  However,  sponsoring  agencies 
may  expand  this  prohibition  as  they  determine  necessary  and  appropriate. 

(d)  A  delineation  of  whether  or  not  the  FFRDC  may  accept  work  from  other 
than  the  sponsor(s).  If  non-sponsor  work  can  be  accepted,  a  delineation 
of  the  procedures  to  be  followed  along  with  any  limitations  as  to  the 
clients  (other  Federal  agencies,  State  or  local  governments,  non-profit  or 
profit  organizations,  etc.)  from  which  work  may  be  accepted.  Limitations 
and  procedures  with  respect  to  responding  to  requests  for  information  as 
to  an  FFRDC’s  capabilities  or  qualifications  are  inherently  a  part  of  the 
"work  for  others"  question  and  will  be  addressed  by  the  sponsoring  agency. 

(2)  Other  Requirements  As  Appropriate 

(a)  When  cost  type  contracts  are  used,  the  sponsor(s)  should  identify  any 
cost  elements  which  will  require  advance  agreement.  Such  items 
may  be,  but  are  not  necessarily  limited  to,  salary  structure, 
depreciation,  various  indirect  costs  such  as  independent  research  and 
development  or  others  as  determined  appropriate  by  the  sponsor(s). 

(b)  Where  fees  are  determined  by  the  sponsor(s)  to  be  appropriate, 
considerations  which  will  affect  their  negotiation  should  be 
identified.  Such  considerations  may  be,  but  are  not  necessarily 
limited  to,  weighted  guidelines,  risks,  use  of  Government  furnished 
property  and  facilities,  needs  or  others  as  determined  appropriate  by 
the  sponsor(s). 
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(c)  other  provisions  as  determined  appropriate  by  the  sponsor(s). 

d.  Changing  the  Basic  Scope  of  an  Existing  FFRDC's  Sponsoring  Agreement. 
In  changing  the  purpose,  mission  and  general  scope  of  effort  to  be  performed  or 
role  of  an  existing  FFRDC  as  set  forth  in  its  sponsoring  agreement  (see  6-<:.(l)(a)), 
the  sponsoring  agency  shall  make  such  changes  consistent  with  its  statutory 
futhcSy  anS5  the8  requirements  ior  estabiishing  a  new  FFRDC  as  set  forth  m 
paragraph  6.b. 

e  Use  of  the  FFRDC  by  the  Sponsor  or  Primary  Sponsor  in  the  Case  o| 
Multiple  Agency  Sponsorship.  The  sponsor,  or  primary  sponsorjn  the  case ^  of 
multiple  sponsorship,  will  enlure  that  all  work  it  places  withitsFFRDC(s)  is  within 
Se  purpose,  mission,  and  general  scope  of  effort  of  the  FFRDC  (paragraph  6.c.) 
and  in  accordance  with  this  Policy  Letter.  This  includes  work  a  sponsoring  agency 
agrees  to  accept  from  a  non-sponsoring  Federal  agency  under  the  protons  of E  The 
Economy  Act  of  1932  (31  USC  1535)  or  other  statutory  authority.  Sponsoring 
agencies  must  comply  with  applicable  procurement  or  assistance :  statutes ^  policies 
and  regulations  for  noncompetitive  actions  before  placing  work  which  is  outside 
the  scope  of  the  sponsor's  contractual  or  sponsoring  agreement  with  an  FFRDC. 

f.  Use  of  an  Existing  FFRDC  by  a  Non-Sponsoring  Federal  Agency  Non¬ 
sponsoring  Federal  agencies  may  use  an  FFRDC  only  if  the  terms  ot  the  fFRDC  s 
sponsoring  agreement  or  contract  permit  work  from  other  than  a  sponsoring 
agency.  Where  use  by  a  non-sponsor  is  permitted  by  the  Sponsoring  Agreement,  the 
work  must  require  the  special  relationship  of  an  FFRDC  as  defined  in  paragraph 
5.c.  and  either  be  treated  as  a  direct  procurement  (action)  or  processed  under  The 
Economy  Act  of  1932  (31  USC  1535)  or  other  statutory  authority.  Work  processed 
under  The  Economy  Act  of  1932  (31  USC  1535)  or  other  statutory  authority  must 
clearly  fall  within  the  purpose,  mission  and  general  scope  of  effort  established  by 
the  sponsoring  agency  for  the  FFRDC  (paragraph  6.C.).  Processing  under  the 
Economy  Act  or  other  statutory  authority  is  subject  to  agreement  by  the  receiving 
agency.  Non-sponsoring  agencies  must  fully  comply  with  procurement  or 
assistance  statutes,  policies  and  regulations  for  non-competitive  actions  prior  to 
placing  work  directly  with  a  specific  FFRDC.  The  FFRDC  must  comp  i  ^"‘before 
procedures  established  by  the  sponsoring  agency  (paragraph  6.c.(l)(d))  before 
accepting  work  from  a  non-sponsoring  Federal  agency. 

p.  Use  of  an  Existing  FFRDC  by  Other  Than  a  Federal  Agency..  Work  from 
other  than  a  Federal  agency  may  be  accepted  only  to  the  extent  permitted  by  the 
sponsoring  agency.  The  FFRDC  must  comply  with  the  procedures  established  by 
the  sponsoring  agency  (paragraph  6.c.(l)(d))  before  accepting  work  from  other  than 
a  Federal  agency. 
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h.  Consulting  Services.  Agencies  sponsoring  FFRDC  work  which  constitutes 
consulting  services,  as  defined  by  0MB  Circular  No.  A-120,  will  comply  with  the 
provisions  of  that  Circular. 

i.  Production/Manufacturing.  FFRDCs  will  not  be  asked  to  perform 
quantity  production  and  manufacturing  work  unless  authorized  by  legislation.  Such 
activities  as  breadboarding,  modeling  or  other  tasks  inherent  to  R<5cD  are 
permissible. 

j.  Periodic  Review.  Prior  to  renewal  of  a  sponsoring  agreement,  agencies 
shall  conduct  a  comprehensive  review  of  their  use  and  need  for  each  FFRDC  that 
they  sponsor.  Where  multiple  agency  sponsorship  exists  this  review  will  be  a 
coordinated  interagency  effort.  When  the  funding  for  an  FFRDC  is  a  specific  line 
item  within  the  sponsoring  agency's  budget,  the  comprehensive  review  may  be  done 
in  conjunction  with  the  budget  process  or  the  review  may  be  done  separately.  The 
sponsoring  agency(s)  shall  apprise  other  agencies  who  use  the  FFRDC  of  the 
scheduled  review  and  afford  them  an  opportunity  to  assume  sponsorship  in  the 
event  the  current  sponsorship  is  determined  no  longer  appropriate.  Final  approval 
to  continue  or  terminate  an  agency's  sponsorship  arrangement  with  a  given  FFRDC 
as  a  result  of  this  review  shall  rest  with  the  head  of  that  sponsoring  agency.  The 
results  of  this  review  will  be  formally  documented.  The  periodic  review  should 
include: 

(1)  An  examination  of  the  agency's  special  technical  'needs  and  mission 
requirements  to  determine  if  and  at  what  level  they  continue  to 
exist. 

(2)  Consideration  of  alternative  sources  to  meet  the  agency's  needs. 
Such  consideration  will  include  compliance  with  the  Notice  and 
Publication  requirements  of  P.L.  98-72  (15  USC  637(e))  prior  to 
renewal  of  the  contract  or  Sponsoring  Agreement  unless  otherwise 
exempted. 

(3)  An  assessment  of  the  efficiency  and  effectiveness  of  the  FFRDC  in 
meeting  the  agency's  needs. 

(4)  An  assessment  of  the  adequacy  of  the  FFRDC  management  in 
assuring  a  cost  effective  operation. 

(5)  A  determination  that  the  guidelines  of  section  6  are  being  satisfied. 

k.  Termination  or  nonrenewal  of  an  FFRDC  Relationship.  When  a  sponsor's 
need  for  the  FFRDC  no  longer  exists,  the  sponsorship  may  be  transferred  to  one  or 
more  Government  agencies,  if  appropriately  justified.  Otherwise  it  shall  be  phased 
out,  the  assets  disposed  of  and  all  liabilities  settled  as  provided  by  the  terms  and 
conditions  of  the  sponsoring  agreement. 
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7.  Action  Reouirements. 

a.  Not  later  than  September  30,  1984,  each  agency  currently  sponsoring  an 
FFRDC  will  review  the  terms  of  its  existing  agreements  with  the  FFRDCs  lor 
compliance  with  ahis  policy  letter.  Where  existing  agreements  do  not  comply  with 
this  policy  letter  the  primary  sponsor  will  develop  a  schedule  to  bring  the 
agreements  into  compliance  not  later  than  the  next  contract  renewal  or  five  years 
from  the  effective  date  of  this  policy  letter,  whichever  comes  first. 

"b.  Where  the  review  required  by  7.a.  reveals  that  a  clear  statement  of  the 
purpose,  mission  and  general  scope  of  effort,  as  described  in  paragraph  6.b.l61  and 
6.c.(l)(a),  does  not  exist,  the  sponsoring  agency  shall  ensure  such  a  statement  is 
developed  not  later  than  September  30, 1984. 

c.  The  primary  sponsor  will  notify  the  Office  of  Science  and  Technology 
Policy  prior  to  designating  any  new  organization  as  an  FFRDC  (paragraph  6  bj, 
changing  the  basic  scope  of  effort  of  an  existing  FFRDC  (paragraph  6.d.)  or 
changing  the  status  of  an  existing  FFRDC  (paragraph  6.k.). 

d.  The  National  Science  Foundation  will  maintain  a  master  Government  list 
of  FFRDCs  based  upon  the  definition  in  this  Policy  Letter. 

e.  FFRDCs  will  be  identified  by  their  primary  sponsors  who  will  provide 
information,  including  funding  data,  on  the  type  of  RicD  being  performed  by  the 
FFRDCs  to  the  National  Science  Foundation  upon  their  request  for  such 

information. 

f.  Each  agency  head  is  responsible  for  ensuring  that  the  provisions  of  this 
policy  are  followed. 

8.  Effective  Date.  The  Policy  Letter  is  effective  (60  days  after  publication  in  the 
Federal  Register). 

9.  Implementation.  Aspects  of  this  policy  letter  requiring  implementation  will  be 
covered  by  the  Department  of  Defense,  the  General  Services  Administration  and 
the  National  Aeronautics  and  Space  Administration  in  the  Federal  Acquisition 
Regulation  not  later  than  180  days  from  the  date  of  this  policy  letter. 
Implementation  will  be  written  so  as  to  be  compatible  with  the  requirements,  as  of 
the  date  of  this  policy  letter,  of  FAR  17.6  "Management  and  Operating  Contracts" 
when  the  arrangement  with  an  FFRDC  constitutes  a  management  and  operating 
contract. 

10.  Information  Contact.  All  questions  or  inquiries  about  this  policy  letter_should 
be  submitted  to  the  Office  of  Management  and  Budget,  Office  of  Federal 
Procurement  Policy,  telephone  (202)  395-6810. 

11.  Sunset  Review  Date.  This  policy  letter  will  be  reviewed  no  later  than  six 
years  after  its  effective  date  for  extension,  modification,  or  rescission. 
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35.017  Federally  Funded  Research  and  Development 
Centers. 

(a)  Policy. 

(1)  This  section  sets  forth  Federal  policy  regarding 
the  establishment,  use,  review,  and  termination  of 
Federally  Funded  Research  and  Development  Centers 
(FFRDC's)  and  related  sponsoring  agreements. 

(2)  An  FFRDC  meets  some  special  long-term 

35-6  (FAC  90-4) 


research  or  development  need  which  cannot  be  met  as 
effectively  by  existing  in-house  or  contractor  resources. 
FFRDC’s  enable  agencies  to  use  private  sector  resources 
to  accomplish  tasks  that  are  integral  to  the  mission  and 
operation  of  the  sponsoring  agency.  An  FFRDC,  in 
order  to  discharge  its  responsibilities  to  the  sponsoring 
agency,  has  access,  beyond  that  which  is  common  to  the 
normal  contractual  relationship,  to  Government  and  sup¬ 
plier  data,  including  sensitive  and  proprietary  data,  and 
to  employees  and  facilities.  The  FFRDC  is  required  to 
conduct  its  business  in  a  manner  befitting  its  special 
relationship  with  the  Government,  to  operate  in  the  pub¬ 
lic  interest  with  objectivity  and  independence,  to  be  free 
from  organizational  conflicts  of  interest,  and  to  have  full 
disclosure  of  its  affaire  to  the  sponsoring  agency.  It  is 
not  the  Government's  intent  that  an  FFRDC  use  its  priv¬ 
ileged  information  or  access  to  facilities  to  compete 
t  with  the  private  sector.  However,  an  FFRDC  may  per¬ 
form  work  for  other  than  the  sponsoring  agency  under 
the  Economy  Act,  or  other  applicable  legislation,  when 
the  work  is  not  otherwise  available  from  the  private  sec¬ 
tor. 

(3)  FFRDCs  are  operated,  managed,  and/or  adminis¬ 
tered  by  either  a  university  or  consortium  of  univereities, 
other  not-for-profit  or  nonprofit  organization,  or  an  indus¬ 
trial  firm,  as  an  autonomous  organization  or  as  an  identi¬ 
fiable  separate  operating  unit  of  a  parent  organization. 

(4)  Long-term  relationships  between  the  Government 
and  FFRDCs  are  encouraged  in  order  to  provide  the 
continuity  that  wQ]  attract  high-quality  personnel  to  the 
FFRDC.  This  relationship  should  be  of  a  type  to 
encourage  the  FFRDC  to  maintain  currency  in  its 
field(s)  of  expertise,  maintain  its  objectivity  and  inde¬ 
pendence,  preserve  its  familiarity  with  the  needs  of  its 
sponsors),  and  provide  a  quick  response  capability. 

(b)  Definitions. 

"Nonsponsor,”  as  used  in  this  section,  means  any  other 
organization,  in  or  outside  of  the  Federal  Government, 
which  funds  specific  work  to  be  performed  by  the  FFRDC 
and  is  not  a  party  to  the  sponsoring  agreement 

“Primary  sponsor,”  as  used  in  this  section,  means  the 
lead  agency  responsible  for  managing,  administering,  or 
monitoring  overall  use  of  the  FFRDC  under  a  multiple 
sponsorship  agreement 

“Special  competency,”  as  used  in  this  section,  means  a 
special  or  unique  capability,  including  qualitative  aspects, 
developed  incidental  to  the  primary  functions  of  the 
FFRDC  to  meet  some  special  need. 

“Sponsor”  means  the  executive  agency  which  manages, 
administers,  monitors,  funds,  and  is  responsible  for  the 
overall  use  of  an  FFRDC.  Multiple  agency  sponsorship  is 
possible  as  long  as  one  agency  agrees  to  act  as  the 
“primary  sponsor.”  In  the  event  of  multiple  sponsors, 
“sponsor”  refers  to  the  primary  sponsor. 
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35.017-1  Sponsoring  agreements. 

(a)  In  order  to  facilitate  a  long-term  relationship 
between  the  Government  and  an  FFRDC,  establish  the 
FFRDC's  mission,  and  ensure  a  periodic  reevaluation  of  the 
FFRDC,  a  written  agreement  of  sponsorship  between  the 
Government  and  the  FFRDC  shall  be  prepared  when  the 
FFRDC  is  established.  The  sponsoring  agreement  may 
take  various  forms;  it  may  be  included  in  a  contract 
between  the  Government  and  the  FFRDC,  or  in  another 
legal  instrument  under  which  an  FFRDC  accomplishes 
effort,  or  it  may  be  in  a  separate  written  agreement. 
Notwithstanding  its  form,  the  sponsoring  agreement  shall 
be  clearly  designated  as  such  by  the  sponsor. 

(b)  While  the  specific  content  of  any  sponsoring  agree¬ 
ment  will  vary  depending  on  the  situation,  the  agreement 
shall  contain,  as  a  minimum,  the  requirements  of  paragraph 

(c)  of  this  subsection.  The  requirements  for,  and  the  con¬ 
tents  of,  sponsoring  agreements  may  be  as  further  specified 
in  sponsoring  agencies'  policies  and  procedures. 

(c)  As  a  minimum,  the  following  requirements  must  be 
addressed  in  either  a  sponsoring  agreement  or  sponsoring 
agencies'  policies  and  procedures: 

(1)  A  statement  of  the  purpose  and  mission  of  the 
FFRDC. 

(2)  Provisions  for  the  orderly  termination  or  nonre¬ 
newal  of  the  agreement,  disposal  of  assets,  and  settle¬ 
ment  of  liabilities.  The  responsibility  for  capitalization 
of  an  FFRDC  must  be  defined  in  such  a  manner  that 
ownership  of  assets  may  be  readily  and  equitably  deter¬ 
mined  upon  termination  of  the  FFRDC’s  relationship 
with  its  sponsors). 

(3)  A  provision  for  the  identification  of  retained 
earnings  (reserves)  and  the  development  of  a  plan  for 
their  use  and  disposition. 

(4)  A  prohibition  against  the  FFRDC  competing 
with  any  non-FFRDC  concern  in  response  to  a  Federal 
agency  request  for  proposal  for  other  than  the  operation 
of  an  FFRDC.  This  prohibition  is  not  required  to  be 
applied  to  any  parent  organization  or  other  subsidiary  of 
the  parent  organization  in  its  non-FFRDC  operations. 
Requests  for  information,  qualifications  or  capabilities 
can  be  answered  unless  otherwise  restricted  by  the  spon¬ 
sor. 

(5)  A  delineation  of  whether  or  not  the  FFRDC  may 
accept  work  from  other  than  the  sponsor(s).  If  nonspon¬ 
sor  work  can  be  accepted,  a  delineation  of  the  proce¬ 
dures  to  be  followed,  along  with  any  limitations  as  to 
the  nonsponsors  from  which  work  can  be  accepted 
(other  Federal  agencies,  State  or  local  governments, 
nonprofit  or  profit  organizations,  etc.). 

(d)  The  sponsoring  agreement  or  sponsoring  agencies’ 
policies  and  procedures  may  also  contain,  as  appropriate, 
other  provisions,  such  as  identification  of — 

(1)  Any  cost  elements  which  will  require  advance 
agreement  if  cost-type  contracts  are  used;  and 


(2)  Considerations  which  will  affect  negotiation  of 

fees  where  payment  of  fees  is  determined  by  the  spon¬ 
sor^)  to  be  appropriate. 

(e)  The  term  of  the  agreement  will  not  exceed  5  years, 
but  can  be  renewed,  as  a  result  of  periodic  review,  in  incre¬ 
ments  not  to  exceed  5  years. 

35.017-2  Establishing  or  changing  an  FFRDC. 

To  establish  an  FFRDC,  or  change  its  basic  purpose  and 
mission,  the  sponsor  shall  ensure  the  following: 

(a)  Existing  alternative  sources  for  satisfying  agency 
requirements  cannot  effectively  meet  the  special  research 
or  development  needs. 

(b)  The  notices  required  for  publication  (see  5.205(b)) 
are  placed  as  required. 

(c)  There  is  sufficient  Government  expertise  available 
to  adequately  and  objectively  evaluate  the  work  to  be  per¬ 
formed  by  the  FFRDC. 

(d)  Tbe  Executive  Office  of  the  President,  Office  of 
Science  and  Technology  Policy,  Washington,  DC  20506,  is 
notified. 

(e)  Controls  are  established  to  ensure  that  the  costs  of  the 
services  being  provided  to  the  Government  are  reasonable. 

(f)  The  basic  purpose  and  mission  of  the  FFRDC  is  stat¬ 
ed  clearly  enough  to  enable  differentiation  between  work 
which  should  be  performed  by  the  FFRDC  and  that  which 
should  be  performed  by  non-FFRDC's. 

(g)  A  reasonable  continuity  in  the  level  of  support  to  the 
FFRDC  is  maintained,  consistent  with  the  agency's  need 
for  the  FFRDC  and  the  terms  of  the  sponsoring  agreement. 

(h)  The  FFRDC  is  operated,  managed,  or  administered 
by  an  autonomous  organization  or  as  an  identifiably  sepa¬ 
rate  operating  unit  of  a  parent  organization,  and  is  required 
to  operate  in  the  public  interest,  free  from  organizational 
conflict  of  interest,  and  to  disclose  its  affairs  (as  an 
FFRDC)  to  the  primary  sponsor. 

(i)  OMB  Circular  A-120  is  complied  with  when 
applicable,  and  quantity  production  or  manufacturing  is  not 
performed  unless  authorized  by  legislation. 

O’)  Approval  is  received  from  the  head  of  the  sponsoring 
agency. 

35.017-3  Using  an  FFRDC. 

(a)  All  work  placed  with  the  FFRDC  must  be  within  the 
purpose,  mission,  general  scope  of  effort,  or  special  compe¬ 
tency  of  the  FFRDC. 

(b)  Where  the  use  of  the  FFRDC  by  a  nonsponsor  is 
permitted  by  the  sponsor,  the  sponsor  shall  be  responsible 
for  compliance  with  paragraph  (a)  of  this  subsection.  The 
nonsponsoring  agency  is  responsible  for  making  the  deter¬ 
mination  required  by  17.502  and  providing  the  documenta¬ 
tion  required  by  17.504(e).  When  permitted  by  the  spon¬ 
sor,  a  Federal  agency  may  contract  directly  with  the 
FFRDC  in  which  case  that  Federal  agency  is  responsible 
for  compliance  with  Part  6. 
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35.017-4  Reviewing  FFRDC's. 

(a)  The  sponsor,  prior  to  extending  the  contract  or 
agreement  with  an  FFRDC,  shall  conduct  a  comprehensive 
review  of  the  use  and  need  for  the  FFRDC.  The  review 
will  be  coordinated  with  any  co-sponsors  and  may  be  per¬ 
formed  in  conjunction  with  the  budget  process.  If  the 
sponsor  determines  that  its  sponsorship  is  no  longer  appro¬ 
priate,  it  shall  apprise  other  agencies  which  use  the  FFRDC 
of  the  determination  and  afford  them  an  opportunity  to 
assume  sponsorship. 

(b)  Approval  to  continue  or  terminate  the  sponsorship 
shall  rest  with  the  head  of  the  sponsoring  agency.  This 
determination  shall  be  based  upon  the  results  of  the  review 
conducted  in  accordance  with  paragraph  (c)  of  this  subsec¬ 
tion. 

(c)  An  FFRDC  review  should  include  the  following: 

(1)  An  examination  of  the  sponsor's  special  technical 
needs  and  mission  requirements  that  are  performed  by 
the  FFRDC  to  determine  if  and  at  what  level  they  con¬ 
tinue  to  exist. 

(2)  Consideration  of  alternative  sources  to  meet  the 
sponsor’s  needs. 

(3)  An  assessment  of  the  efficiency  and  effective¬ 
ness  of  the  FFRDC  in  meeting  the  sponsor's  needs, 
including  the  FFRDCs  ability  to  maintain  its  objectivi¬ 
ty,  independence,  quick  response  capability,  currency  in 
its  field(s)  of  expertise,  and  familiarity  with  the  needs  of 
its  sponsor. 

(4)  An  assessment  of  the  adequacy  of  the  FFRDC 
management  in  ensuring  a  cost-effective  operation. 

(5)  A  determination  that  the  criteria  for  establishing 
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the  FFRDC  continue  to  be  satisfied  and  that  the  spon¬ 
soring  agreement  is  in  compliance  with  35.017-1. 

35.017-5  Terminating  an  FFRDC. 

When  a  sponsor's  need  for  the  FFRDC  no  longer  exists, 
the  sponsorship  may  be  transferred  to  one  or  more 
Government  agencies,  if  appropriately  justified.  If  the 
FFRDC  is  not  transferred  to  another  Government  agency,  it 
shall  be  phased  out 

35.017-6  Master  list  of  FFRDC's. 

The  National  Science  Foundation  (NSF)  maintains  a 
master  Government  list  of  FFRDC's.  Primary  sponsors 
will  provide  information  on  each  FFRDC,  including  spon¬ 
soring  agreements,  mission  statements,  funding  data,  and 
type  of  R&D  being  performed,  to  the  NSF  upon  its  request 
for  such  information. 

35.017-7  Limitation  on  the  creation  of  new  FFRDC's. 

Pursuant  to  10  U.S.C.  2367,  the  Secretary  of  Defense, 
the  Secretary  of  the  Army,  the  Secretary  of  the  Navy,  the 
Secretary  of  the  Air  Force,  the  Secretary  of  Transportation, 
and  the  Administrator  of  the  National  Aeronautics  and 
Space  Administration  may  not  obligate  or  expend  amounts 
appropriated  to  the  Department  of  Defense  for  purposes  of 
operating  an  FFRDC  that  was  not  in  existence  before  June 
2,  1986,  until  (a)  the  head  of  the  agency  submits  to 
Congress  a  report  with  respect  to  such  center  that  describes 
the  purpose,  mission,  and  general  scope  of  effort  of  the 
center;  and  (b)  a  period  of  60  days,  beginning  on  the  date 
such  report  is  received  by  Congress,  has  elapsed. 
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Appendix  O:  USD(A&T)  Letter  dated  April  22, 1997 

THE  UNDER  SECRETARY  OF  DEFENSE 
3010  DEFENSE  PENTAGON 
WASHINGTON,  D.C.  20301-3010 


APR  2  2  1997 


MEMORANDUM  FOR  THE  CHAIRMAN,  DEFENSE  SCIENCE  BOARD 

SUBJECT:  Report  of  the  Defense  Science  Board  Independent  Advisory  Task 
Force  on  Federally  Funded  Research  and  Development  Centers  and 
University  Affiliated  Research  Centers 


This  memorandum  provides  DoD  comments  on  the  January  1997  report  of  the  Defense 
Science  Board’s  (DSB)  Independent  Advisory  Task  Force  on  Federally  Funded  Research  and 
Development  Centers  (FFRDCs)  and  University  Affiliated  Research  Centers  (UARCs).  The 
comments  are  based  on  reviews  by  the  senior  users  of  FFRDC  work,  including  the  Service  Vice 
Chiefs  and  Acquisition  Executives.  The  Director  of  Defense  Research  and  Engineering 
coordinated  the  review  effort.  Also,  I  personally  reviewed  the  report  and  fully  support  the 
observations  and  conclusions  that  follow. 

The  reviewers  disagreed  with  the  report’s  findings  and  recommendations.  For  example, 
the  Army  Vice  Chief  of  Staff  and  Assistant  Secretaiy  (Research,  Development  and  Acquisition) 
commented  that  “the  Army  non-concurs  with  the  recommendations  contained  in  the  subject 
report. .  .The  Army  would  welcome  constructive  recommendations  relating  to  its  management 
policies  and  procedures  in  order  to  continue  developing  the  trust. .  .that  FFRDCs  are  being 
properly  managed  and  utilized.  We  believe  that  this  report  fails  to  assist  us  in  achieving  that 
objective.”  Similarly,  the  Air  Force  Vice  Chief  of  Staff  and  Assistant  Secretaiy  (Acquisition)  said 
“we  completely  disagree  with  the  entire  premise  of  the  task  force  s  findings  and 
recommendations.  The  key  findings  are  not  well  substantiated  and  contradict  our  ongoing 
experience. .  .We  recommend  that  the  Department  ignore  the  recommendations  of  the  task  force 
and  disband  the  group.”  Finally,  the  OSD  Principal  Deputy,  PA&E  said  “I  have  reviewed  the 
DSB  task  force  report  and  disagree  with  its  findings  and  recommendations.  The  key  findings  do 
not  reflect  the  importance  of  study  and  analysis  (S&A)  centers  to  P  A&E.  In  addition,  the  report 
contradicts  the  findings  and  recommendations  of  the  Defense  Science  Board’s  1995  review  of 
FFRDCs,  without  a  full  explanation.” 

In  my  experience,  this  report  is  disappointing.  I  say  that  not  because  I  disagree  with  the 
findings  --  which  I  do  ~  but  because  the  task  force  did  not  examine  the  subjects  we  requested  it 
to,  and  because  the  task  force  seemed  ill-informed  on  the  issues  that  it  did  address. 

We  established  this  task  force  to  review  and  advise  DoD  on  management  practices  for 
FFRDCs  and  UARCs.  We  were  looking  for  suggestions  on  how  to  improve  the  ways  in  which 
DoD  sponsors  and  users  interact  with  FFRDCs  and  UARCs  to  ensure  that  we  get  the  best  value 


from  these  organizations.  Instead,  the  task  force  focused  on  broad  management  policy  issues  for 
FFRDCs.  Many  of  the  task  force’s  findings  are  not  well  substantiated  or  documented,  and  they 
contradict  my  own  experiences  and  the  experiences  of  the  majority  of  sponsors  and  users  of 
FFRDC  research  products  throughout  DoD.  They  also  ignore  or  are  inconsistent  with  the  results 
of  a  recent  internal  review  of  FFRDCs,  as  well  as  the  previous  DSB  Task  Force  Report  on 
FFRDCs  of  April  1995.  Overall,  it  is  my  sense  that  the  current  task  force  was  neither  established 
nor  staffed  to  address  broad  policy  issues  regarding  FFRDCs,  and  its  findings  on  these  subjects 
are  questionable. 

Several  of  the  findings  seem  to  have  followed  from  the  task  force’s  misunderstanding  of  why 
DoD  maintains  FFRDCs.  The  task  force  wrongly  assumed  that  FFRDCs  are  maintained  only  to 
conduct  work  for  which  there  are  no  alternative  sources.  Based  on  this  assumption,  the  task  force 
recommended  that  DoD  eliminate  all  research  at  FFRDCs  whenever  it  can  be  shown  that  alternative 
sources  exist  among  for-profit  firms.  But  that  misses  the  point.  FFRDCs  are  critical  national  assets 
because:  (1)  they  maintain  long-term  strategic  relationships  with  their  DoD  sponsors;  (2)  they 
perform  research,  development  and  analytic  tasks  that  are  integral  to  the  mission  and  operations  of 
DoD  sponsors;  (3)  they  maintain  “core”  competencies  in  areas  important  to  their  sponsors, 
performing  high  quality,  objective  work  that  could  not  be  carried  out  as  effectively  by  other  private 
or  public  organizations;  and  (4)  they  operate  in  the  public  interest,  free  from  real  or  perceived 
conflicts  of  interest.  The  DSB  task  force  largely  ignored  these  reasons  DoD  maintains  FFRDCs,  and 
they  reached  flawed  conclusions  as  a  result. 

I  might  note  that  maintaining  long-term  strategic  relationships  is  one  area  where  DoD  has 
been  in  front  of  the  commercial  sector  in  its  acquisition  practices.  Successful  commercial  firms  are 
moving  increasingly  in  the  direction  of  establishing  long-term,  strategic  relationships  with  their  key 
suppliers.  They  have  found  the  result  is  often  a  higher  quality  product,  at  lower  overall  costs,  in 
contrast  to  the  previous  practice  of  changing  suppliers  based  on  recurring,  short-term  low  bids. 

DoD  has  long  realized  this  benefit  from  FFRDCs. 

I  am  not  arguing  that  competition  is  inappropriate.  The  Department  uses  competitive 
processes  to  obtain  the  overwhelming  majority  of  the  goods  and  services  that  we  require.  But  there 
are  some  circumstances  and  some  kinds  of  work,  for  which  the  value  provided  by  a  strategic 
relationship  far  outweighs  the  potential  gains  of  competition.  The  task  force  did  not  have  sufficient 
familiarity  with  FFRDCs  to  recognize  that  these  organizations  represent  one  of  those  circumstances. 

Two  other  task  force  misconceptions  are  worth  noting.  First,  the  task  force  implied  that 
DoD  customers  were  unaware  of  the  benefits  of  alternative  sources.  Nothing  could  be  further  from 
the  truth.  DoD  now  uses  for-profit  industry  to  conduct  most  of  its  RDT&E  work  of  all  kinds,  with 
less  than  3%  going  to  the  Systems  Engineering  and  Studies  and  Analyses  FFRDCs  covered  in  this 
report.  FFRDC  sponsors  are  quite  familiar  with  the  capabilities  of  alternative  sources,  and  they  use 
them  when  it  is  appropriate  to  do  so. 

Second,  the  task  force  was  concerned  that  FFRDCs  might  not  be  able  to  keep  pace  with  fast- 
moving  technologies  —  particularly  in  software  and  communications  --  and  that  DoD  might  make  an 
investment  mistake  as  a  result.  Here,  the  task  force  seemed  to  misunderstand  how  sponsors  use 


FFRDCs,  and  how  FFRDCs  fit  into  the  overall  mix  of  DoD  research  capabilities.  It’s  obvious  that 
the  Department  cannot,  and  does  not,  rely  on  a  handful  of  organizations  for  its  technology 
innovation.  Because  of  their  access  to  proprietary  information  from  numerous  commercial  firms, 
FFRDCs  are  often  used  to  help  assess  the  value,  costs,  and  risks  of  technology  developments.  Thus, 
FFRDCs  act  as  trusted  advisors  helping  DoD  incorporate  the  best  technologies  developed  in  the 
commercial  sector. 

Finally,  the  Air  Force’s  recent  experience  with  the  proposed  Aerospace  -  SAIC  merger 
provides  perspective  on  the  task  force’s  recommendations.  The  Air  Force  considered  a  variety  of 
alternatives  similar  to  what  the  task  force  proposes  -  such  as  limiting  the  FFRDCs  to  a  stricter 
definition  of  their  core  competencies  and  competing  the  remaining  work.  The  Air  Force  concluded 
that  such  a  change  would  “negatively  impact  the  trusted  agent  role  the  Aerospace  FFRDC  has  served 
for  over  36  years  and  is  not  in  the  best  interest  of  the  U.S.  Government.”  In  the  Air  Force’s  view, 
separating  pieces  of  Aerospace’s  work  for  competition  would  reduce  the  quality  and  value  of  the 
overall  space  support  effort.  Although  discrete  pieces  of  Aerospace’s  work  could  be  performed  by 
individual  for-profit  contractors,  it  is  the  unique  synergism  between  all  elements  of  the  FFRDC  that 
makes  Aerospace  a  critical  asset.  Other  for-profit  firms  that  had  routinely  shared  proprietary 
information  with  Aerospace  opposed  the  merger  as  well,  reflecting  concerns  about  conflicts  of 
interest  that  would  arise  if  the  Air  Force  attempted  to  break-up  the  unique  FFRDC  mission  and  offer 
some  of  it  up  for  competition.  Similar  reviews  of  the  other  FFRDCs  -  many  of  which  have  broader 
charters  and  more  extensive  access  to  sensitive  information  than  Aerospace  —  would  produce  similar 
results. 


In  summary,  I  believe  --  and  this  belief  is  held  widely  in  the  Department,  both  by  civilian  and 
military  leaders  -  that  FFRDCs  are  doing  high-quality,  high-value  technical  and  analytical  work  that 
could  not  be  provided  as  effectively  by  other  means.  I  also  believe  that  we  have  put  in  place  a  sound 
management  approach.  FFRDCs  are  sized  consistent  with  essential  sponsor  requirements, 
acquisition  reform  initiatives,  and  defense  strategies  and  budgets.  The  annual  workload  is  based  on 
DoD  needs  for  the  individual  core  competencies  maintained  at  each  FFRDC,  and  we  have  a 
established  a  5-year  plan  for  the  required  staff-years  of  technical  effort.  We  intend  to  continue  to 
review  and  update  this  plan.  We  have  informed  Congress  of  these  steps,  and  we  will  continue  to 
keep  the  relevant  Committees  abreast  of  our  management  plans  and  actions.  Based  on  my 
conversations  with  members  and  staff,  I  believe  the  Congress  will  support  these  DoD  initiatives. 

The  problem  is  that  the  DSB  task  force  went  beyond  its  mandate  and  the  conclusions  in  those 
areas  were  flawed  due  to  misperceptions  regarding  the  FFRDC  concept  and  what  constitutes 
appropriate  work.  The  task  force  has  proposed  radical  solutions  to  problems  that  don’t  exist. 

Rather  than  helping  us  figure  out  ways  to  manage  these  critical  resources  most  effectively,  the  task 
force  proposes  to  dismantle  high-quality,  high-value  defense  capabilities  that  are  serving  the  national 
interest.  With  the  exception  of  several  suggestions  regarding  specific  management  practices,  the 
task  force’s  recommendations  have  little  value.  To  ensure  that  the  Department’s  observations  and 
conclusion  are  clearly  represented  in  the  report,  I  have  also  asked  the  Executive  Director  of  the  DSB 
to  include  this  memorandum  in  the  “printed  for  distribution”  version. 


Paul  6.  Kaminski 


